demus/openhab-addons
1
0
Fork 0
mirror of https://github.com/danieldemus/openhab-addons synced 2026-07-27 19:50:36 +02:00
Code Issues Packages Projects Releases Wiki Activity

[timescaledb] Add metadata tag support and JSONB config storage (fixes #20460) (#20464)

Browse Source 
* feat: Enhance TimescaleDB schema with value and metadata columns

Signed-off-by: René Ulbricht <rene_ulbricht@outlook.com>
This commit is contained in:
ulbi
2026-03-28 17:54:59 +01:00
committed by GitHub
parent 98c5650ab2
commit b4846f5e45
14 changed files with 1018 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files
bundles/org.openhab.persistence.timescaledb/AGENTS.md
+41 -4
View File
@@ -49,6 +49,8 @@ CREATE TABLE item_meta (
id SERIAL PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
label TEXT,
value TEXT,
metadata JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
@@ -65,6 +67,36 @@ SELECT create_hypertable('items', 'time');
CREATE INDEX ON items (item_id, time DESC);
```
### `item_meta.value` and `item_meta.metadata`
`value TEXT` stores `metadata.getValue()` — a free user-defined string (measurement label, filter tag, etc.).
`metadata JSONB` stores the **complete** `metadata.getConfiguration()` map serialized as JSON — unfiltered,
including reserved keys (`aggregation`, `downsampleInterval`, `retainRawDays`, `retentionDays`) and any
user-defined tags (`location`, `kind`, etc.).
```
Number:Temperature MySensor {
timescaledb="sensor.temperature" [ aggregation="AVG", downsampleInterval="1h",
location="living_room", kind="sensor" ]
}
-- item_meta.value = 'sensor.temperature'
-- item_meta.metadata = '{"aggregation":"AVG","downsampleInterval":"1h","location":"living_room","kind":"sensor"}'
```
Grafana can filter via JSONB operators: `WHERE metadata->>'location' = 'living_room'`
When no value/config is set, both columns are `NULL`.
**Migration:** On startup `TimescaleDBSchema.initialize()` runs a single DO-block that adds both columns atomically:
```sql
ALTER TABLE item_meta
ADD COLUMN IF NOT EXISTS value TEXT,
ADD COLUMN IF NOT EXISTS metadata JSONB;
```
`IF NOT EXISTS` makes the statement idempotent. A `lock_timeout` of 5 s prevents blocking `@Activate` indefinitely; on timeout a WARNING is logged and the migration is retried on the next startup.
### Why `unit` is per row, not in `item_meta`
A `QuantityType` unit can change over time (sensor reconfiguration, firmware update, etc.). Storing it in `item_meta` would corrupt historical reads. The unit is stored with each measurement and read back from the row when reconstructing `QuantityType` states.
@@ -134,17 +166,22 @@ private Optional<Metadata> getItemMetadata(String itemName) {
```
`Metadata` has:
- `getValue()` → main value string, e.g. `"AVG"`, `"MAX"`, `"MIN"`, `"SUM"`, or `""` (no aggregation)
- `getConfiguration()` → `Map<String, Object>` with keys like `"downsampleInterval"`, `"retainRawDays"`, `"retentionDays"`
- `getValue()` → user-defined string (e.g. `"sensor.temperature"`), stored in `item_meta.value`
- `getConfiguration()` → `Map<String, Object>` with keys like `"aggregation"`, `"downsampleInterval"`, `"retainRawDays"`, `"retentionDays"`, plus user-defined tags
### Metadata format (configured by users in .items files)
```java
Number:Temperature MySensor {
timescaledb="AVG" [ downsampleInterval="1h", retainRawDays="5", retentionDays="365" ]
timescaledb="sensor.temperature" [ aggregation="AVG", downsampleInterval="1h",
retainRawDays="5", retentionDays="365", location="living_room" ]
}
```
- `getValue()` = user-defined string stored in `item_meta.value` (e.g. measurement label for Grafana)
- `aggregation` in config = downsampling function (replaces the old `getValue()` = `"AVG"` pattern)
- All config keys are stored unfiltered as JSONB in `item_meta.metadata`
### Parsing the metadata
```java
@@ -249,7 +286,7 @@ Location: `src/test/java/org/openhab/persistence/timescaledb/internal/`
- `TimescaleDBMetadataServiceTest` — parsing of metadata values and config keys
- `TimescaleDBDownsampleJobTest` — SQL generation for aggregation/delete, interval allowlist validation
Run with `mvn test` — last result: **183 tests, 0 failures** (2026-03-13).
Run with `mvn test` — last result: **228 tests, 0 failures** (2026-03-28).
### Integration Tests (requires Docker + TimescaleDB)
bundles/org.openhab.persistence.timescaledb/README.md
+60 -17
View File
@@ -27,7 +27,7 @@ CREATE EXTENSION IF NOT EXISTS timescaledb;
## Database Schema
The service **creates all tables automatically on startup** — no manual DDL required.
Item states are stored in a single hypertable `items` (columns: `time`, `item_id`, `value`, `string`, `unit`, `downsampled`) and a name-lookup table `item_meta`.
Item states are stored in a single hypertable `items` (columns: `time`, `item_id`, `value`, `string`, `unit`, `downsampled`) and a name-lookup table `item_meta` (columns: `id`, `name`, `label`, `value`, `metadata`).
## State Type Mapping
@@ -77,36 +77,44 @@ Items {
}
```
## Per-Item Downsampling
## Per-Item Downsampling and Metadata Tags
Downsampling is configured **per item** via item metadata in the `timescaledb` namespace.
Per-item behaviour is configured via item metadata in the `timescaledb` namespace.
### Metadata format
```text
timescaledb="<operation>" [downsampleInterval="<interval>", retainRawDays="<n>", retentionDays="<n>"]
timescaledb="<label>" [ aggregation="<fn>", downsampleInterval="<interval>",
retainRawDays="<n>", retentionDays="<n>", <custom-tag>="<value>", ... ]
```
| Metadata key | Values | Description |
|----------------------|--------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|
| value (main) | `AVG`, `MAX`, `MIN`, `SUM`, or `" "` | Aggregation function. Use a single space `" "` for retention-only (no downsampling). openHAB rejects a truly empty value, so a space is required. |
| `downsampleInterval` | e.g. `1h`, `15m`, `1d` | Time bucket size for aggregation. Required when value is an aggregation function. |
| `retainRawDays` | integer, default `5` | Keep raw data for N days before replacing with aggregated rows. |
| `retentionDays` | integer, default `0` | Drop all data (raw + downsampled) older than N days. `0` = off. |
| Metadata key | Values / default | Description |
|----------------------|------------------------------|---------------------------------------------------------------------------------------------------------------|
| value (main) | any string | User-defined label stored in `item_meta.value`. Leave blank (single space `" "`) if only retention is needed. |
| `aggregation` | `AVG`, `MAX`, `MIN`, `SUM` | Downsampling aggregation function. Omit if no downsampling is needed. |
| `downsampleInterval` | e.g. `1h`, `15m`, `1d` | Time bucket size for aggregation. Required when `aggregation` is set. |
| `retainRawDays` | integer, default `5` | Keep raw data for N days before replacing with aggregated rows. |
| `retentionDays` | integer, default `0` | Drop all data older than N days. `0` = disabled. |
| custom tags | any key=value pairs | Stored unfiltered as JSONB in `item_meta.metadata`. Queryable via Grafana/SQL JSONB operators. |
The **entire config map** (all keys including `aggregation`, `downsampleInterval`, etc.) is stored as JSONB in `item_meta.metadata`, enabling flexible SQL/Grafana filtering.
### Configuration in `.items` files
```java
// Downsampling + custom tags for Grafana filtering
Number:Temperature Sensor_Temperature_Living "Living Room [%.1f °C]" {
timescaledb="AVG" [ downsampleInterval="1h", retainRawDays="5" ]
timescaledb="sensor.temperature" [ aggregation="AVG", downsampleInterval="1h",
retainRawDays="5", location="living_room", kind="temperature" ]
}
Number:Power Meter_Power_House "House Power [%.1f W]" {
timescaledb="AVG" [ downsampleInterval="15m", retainRawDays="3", retentionDays="365" ]
timescaledb="meter.power" [ aggregation="AVG", downsampleInterval="15m",
retainRawDays="3", retentionDays="365" ]
}
Number:Energy Meter_Energy_House "House Energy [%.3f kWh]" {
timescaledb="SUM" [ downsampleInterval="1h", retainRawDays="7" ]
timescaledb="meter.energy" [ aggregation="SUM", downsampleInterval="1h", retainRawDays="7" ]
}
// Retention-only: no downsampling, just drop data older than 30 days.
@@ -118,12 +126,12 @@ Number:Temperature Sensor_Temp_Outdoor {
### Configuration in mainUI
**Downsampling + Retention:**
**Downsampling + Retention + Tags:**
`Item → Metadata → Add Metadata → Enter namespace "timescaledb"`:
- Value: `AVG`
- Additional config: `downsampleInterval=1h`, `retainRawDays=5`, `retentionDays=365`
- Value: `sensor.temperature` (or any descriptive label)
- Additional config: `aggregation=AVG`, `downsampleInterval=1h`, `retainRawDays=5`, `retentionDays=365`, `location=living_room`
**Retention-only (no downsampling):**
@@ -204,7 +212,9 @@ This works independently of downsampling: an item can have `retentionDays` set w
## Grafana Integration
TimescaleDB works natively with the Grafana PostgreSQL data source:
TimescaleDB works natively with the Grafana PostgreSQL data source.
### Query by item name
```sql
-- Raw + downsampled data for a sensor (last 24 h)
@@ -220,6 +230,39 @@ GROUP BY 1
ORDER BY 1;
```
### Filter by label (`item_meta.value`)
```sql
-- All items labelled "sensor.temperature" (last 24 h)
SELECT
time_bucket('5 minutes', time) AS time,
item_meta.name AS sensor,
AVG(value) AS temperature
FROM items
JOIN item_meta ON items.item_id = item_meta.id
WHERE item_meta.value = 'sensor.temperature'
AND time > NOW() - INTERVAL '24 hours'
GROUP BY 1, 2
ORDER BY 1;
```
### Filter by custom tag (`item_meta.metadata` JSONB)
```sql
-- All temperature sensors in the living room
SELECT
time_bucket('5 minutes', time) AS time,
item_meta.name AS sensor,
AVG(value) AS temperature
FROM items
JOIN item_meta ON items.item_id = item_meta.id
WHERE item_meta.metadata->>'location' = 'living_room'
AND item_meta.metadata->>'kind' = 'temperature'
AND time > NOW() - INTERVAL '24 hours'
GROUP BY 1, 2
ORDER BY 1;
```
## Differences from JDBC Persistence
| Feature | JDBC Persistence | TimescaleDB Persistence |
bundles/org.openhab.persistence.timescaledb/pom.xml
+8 -1
View File
@@ -15,7 +15,7 @@
<name>openHAB Add-ons :: Bundles :: Persistence Service :: TimescaleDB</name>
<properties>
<bnd.importpackage>!com.codahale.metrics.*,!io.prometheus.*,!org.checkerframework.*,!org.jetbrains.annotations.*,!org.hibernate.*,!waffle.windows.auth.*,!org.osgi.service.jdbc.*,!com.sun.jna.*,!javassist.*</bnd.importpackage>
<bnd.importpackage>!com.codahale.metrics.*,!io.prometheus.*,!org.checkerframework.*,!org.jetbrains.annotations.*,!org.hibernate.*,!waffle.windows.auth.*,!org.osgi.service.jdbc.*,!com.sun.jna.*,!javassist.*,!com.google.errorprone.annotations.*,!sun.misc.*</bnd.importpackage>
<postgresql.version>42.7.9</postgresql.version>
<hikari.version>5.1.0</hikari.version>
</properties>
@@ -34,6 +34,13 @@
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>${gson.version}</version>
<scope>compile</scope>
</dependency>
<!-- Test dependencies -->
<dependency>
<groupId>org.testcontainers</groupId>
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBMetadataService.java
+71 -15
View File
@@ -14,6 +14,7 @@ package org.openhab.persistence.timescaledb.internal;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import org.eclipse.jdt.annotation.NonNullByDefault;
@@ -27,19 +28,28 @@ import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.gson.Gson;
/**
* Reads and parses per-item downsampling configuration from the {@link MetadataRegistry}
* using the {@code timescaledb} namespace.
* Reads per-item configuration from the {@link MetadataRegistry} using the {@code timescaledb} namespace.
*
* <p>
* Example item metadata:
*
*
* <pre>
* Number:Temperature MySensor {
* timescaledb="AVG" [ downsampleInterval="1h", retainRawDays="5", retentionDays="365" ]
* timescaledb="sensor.temperature" [ aggregation="AVG", downsampleInterval="1h", retainRawDays="5",
* retentionDays="365", kind="sensor", location="living_room" ]
* }
* </pre>
*
* <ul>
* <li>{@code getValue()} — user-defined string (measurement label / filter tag), stored in
* {@code item_meta.value}</li>
* <li>{@code getConfiguration()} — full config map stored as JSONB in {@code item_meta.metadata}; reserved keys:
* {@code aggregation}, {@code downsampleInterval}, {@code retainRawDays}, {@code retentionDays}</li>
* </ul>
*
* @author René Ulbricht - Initial contribution
*/
@NonNullByDefault
@@ -47,6 +57,7 @@ import org.slf4j.LoggerFactory;
public class TimescaleDBMetadataService {
private static final Logger LOGGER = LoggerFactory.getLogger(TimescaleDBMetadataService.class);
private static final Gson GSON = new Gson();
/** The metadata namespace used by this persistence service. */
public static final String METADATA_NAMESPACE = "timescaledb";
@@ -95,13 +106,61 @@ public class TimescaleDBMetadataService {
return result;
}
/**
* Returns the user-defined value string from {@code metadata.getValue()}, stored verbatim in
* {@code item_meta.value}. Returns {@code null} if no metadata is configured or the value is blank.
*
* <p>
* Example: {@code timescaledb="sensor.temperature" [...]} → returns {@code "sensor.temperature"}.
*
* @param itemName The item name.
* @return The value string, or {@code null}.
*/
public @Nullable String getMetadataValueString(String itemName) {
MetadataKey key = new MetadataKey(METADATA_NAMESPACE, itemName);
@Nullable
Metadata metadata = metadataRegistry.get(key);
if (metadata == null) {
return null;
}
String v = metadata.getValue();
return v.isBlank() ? null : v;
}
/**
* Returns the full {@code getConfiguration()} map serialized as a JSON string, suitable for storage
* in {@code item_meta.metadata} (JSONB column). Returns {@code null} if no metadata is configured
* or the config map is empty.
*
* <p>
* All config keys are stored unfiltered, including reserved keys ({@code aggregation},
* {@code downsampleInterval}, {@code retainRawDays}, {@code retentionDays}) and any user-defined tags.
*
* @param itemName The item name.
* @return JSON string of the config map, or {@code null}.
*/
public @Nullable String getMetadataConfigJson(String itemName) {
MetadataKey key = new MetadataKey(METADATA_NAMESPACE, itemName);
@Nullable
Metadata metadata = metadataRegistry.get(key);
if (metadata == null) {
return null;
}
Map<String, Object> config = metadata.getConfiguration();
if (config.isEmpty()) {
return null;
}
return GSON.toJson(config);
}
private Optional<DownsampleConfig> parseConfig(String itemName, Metadata metadata) {
String functionStr = metadata.getValue();
var config = metadata.getConfiguration();
Object aggObj = config.get("aggregation");
String functionStr = aggObj != null ? aggObj.toString().trim() : "";
if (functionStr.isBlank()) {
// No aggregation function — check for retention-only config.
// Note: openHAB requires a non-empty metadata value, so use a single space (" ")
// in item files and the UI when you only want retention without downsampling.
int retentionDays = getInt(metadata.getConfiguration(), "retentionDays", DEFAULT_RETENTION_DAYS);
// No aggregation function — retention-only config (retentionDays without downsampling).
int retentionDays = getInt(config, "retentionDays", DEFAULT_RETENTION_DAYS);
if (retentionDays < 0) {
LOGGER.warn("Item '{}': retentionDays must be >= 0, ignoring negative value {}", itemName,
retentionDays);
@@ -123,11 +182,9 @@ public class TimescaleDBMetadataService {
return Optional.empty();
}
var config = metadata.getConfiguration();
String intervalStr = getString(config, "downsampleInterval", null);
if (intervalStr == null || intervalStr.isBlank()) {
LOGGER.warn("Item '{}': timescaledb metadata has function '{}' but no downsampleInterval — skipping",
LOGGER.warn("Item '{}': timescaledb metadata has aggregation '{}' but no downsampleInterval — skipping",
itemName, functionStr);
return Optional.empty();
}
@@ -156,13 +213,12 @@ public class TimescaleDBMetadataService {
return Optional.of(result);
}
private static @Nullable String getString(java.util.Map<String, Object> config, String key,
@Nullable String defaultValue) {
private static @Nullable String getString(Map<String, Object> config, String key, @Nullable String defaultValue) {
Object val = config.get(key);
return val != null ? val.toString() : defaultValue;
}
private static int getInt(java.util.Map<String, Object> config, String key, int defaultValue) {
private static int getInt(Map<String, Object> config, String key, int defaultValue) {
Object val = config.get(key);
if (val == null) {
return defaultValue;
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBPersistenceService.java
+47 -6
View File
@@ -27,10 +27,13 @@ import java.util.concurrent.TimeUnit;
import org.eclipse.jdt.annotation.NonNullByDefault;
import org.eclipse.jdt.annotation.Nullable;
import org.openhab.core.common.ThreadPoolManager;
import org.openhab.core.common.registry.RegistryChangeListener;
import org.openhab.core.config.core.ConfigurableService;
import org.openhab.core.items.Item;
import org.openhab.core.items.ItemNotFoundException;
import org.openhab.core.items.ItemRegistry;
import org.openhab.core.items.Metadata;
import org.openhab.core.items.MetadataRegistry;
import org.openhab.core.persistence.FilterCriteria;
import org.openhab.core.persistence.HistoricItem;
import org.openhab.core.persistence.ModifiablePersistenceService;
@@ -69,7 +72,7 @@ import com.zaxxer.hikari.HikariDataSource;
TimescaleDBPersistenceService.class }, configurationPid = "org.openhab.timescaledb", configurationPolicy = ConfigurationPolicy.REQUIRE, property = Constants.SERVICE_PID
+ "=org.openhab.timescaledb")
@ConfigurableService(category = "persistence", label = "TimescaleDB Persistence Service", description_uri = TimescaleDBPersistenceService.CONFIG_URI)
public class TimescaleDBPersistenceService implements ModifiablePersistenceService {
public class TimescaleDBPersistenceService implements ModifiablePersistenceService, RegistryChangeListener<Metadata> {
static final String CONFIG_URI = "persistence:timescaledb";
static final String CONFIGURATION_PID = "org.openhab.timescaledb";
@@ -84,6 +87,7 @@ public class TimescaleDBPersistenceService implements ModifiablePersistenceServi
private final Map<String, Integer> itemIdCache = new ConcurrentHashMap<>();
private final ItemRegistry itemRegistry;
private final MetadataRegistry metadataRegistry;
private final TimescaleDBMetadataService metadataService;
private @Nullable HikariDataSource dataSource;
@@ -92,15 +96,18 @@ public class TimescaleDBPersistenceService implements ModifiablePersistenceServi
@Activate
public TimescaleDBPersistenceService(final @Reference ItemRegistry itemRegistry,
final @Reference MetadataRegistry metadataRegistry,
final @Reference TimescaleDBMetadataService metadataService) {
this.itemRegistry = itemRegistry;
this.metadataRegistry = metadataRegistry;
this.metadataService = metadataService;
}
/** Package-private constructor for unit tests — skips OSGi activation, allows injecting a DataSource. */
TimescaleDBPersistenceService(ItemRegistry itemRegistry, TimescaleDBMetadataService metadataService,
@Nullable HikariDataSource dataSource) {
TimescaleDBPersistenceService(ItemRegistry itemRegistry, MetadataRegistry metadataRegistry,
TimescaleDBMetadataService metadataService, @Nullable HikariDataSource dataSource) {
this.itemRegistry = itemRegistry;
this.metadataRegistry = metadataRegistry;
this.metadataService = metadataService;
this.dataSource = dataSource;
}
@@ -159,6 +166,7 @@ public class TimescaleDBPersistenceService implements ModifiablePersistenceServi
TimeUnit.DAYS.toSeconds(1), TimeUnit.SECONDS);
LOGGER.info("Downsampling job scheduled: first run in {}s, then every 24h", initialDelay);
metadataRegistry.addRegistryChangeListener(this);
LOGGER.info("TimescaleDB persistence service activated");
}
@@ -180,6 +188,7 @@ public class TimescaleDBPersistenceService implements ModifiablePersistenceServi
@Deactivate
public void deactivate() {
LOGGER.debug("Deactivating TimescaleDB persistence service");
metadataRegistry.removeRegistryChangeListener(this);
itemIdCache.clear();
ScheduledFuture<?> job = downsampleJob;
@@ -251,6 +260,10 @@ public class TimescaleDBPersistenceService implements ModifiablePersistenceServi
String name = alias != null ? alias : item.getName();
@Nullable
String label = item.getLabel();
@Nullable
String valueStr = metadataService.getMetadataValueString(name);
@Nullable
String metadataJson = metadataService.getMetadataConfigJson(name);
HikariDataSource ds = dataSource;
if (ds == null) {
@@ -259,7 +272,7 @@ public class TimescaleDBPersistenceService implements ModifiablePersistenceServi
}
try (Connection conn = ds.getConnection()) {
int itemId = getOrCreateItemId(conn, name, label);
int itemId = getOrCreateItemId(conn, name, label, valueStr, metadataJson);
TimescaleDBQuery.insert(conn, itemId, date, row);
} catch (SQLException e) {
LOGGER.error("Failed to store item '{}': {}", name, e.getMessage(), e);
@@ -375,16 +388,44 @@ public class TimescaleDBPersistenceService implements ModifiablePersistenceServi
}
}
// -------------------------------------------------------------------------
// RegistryChangeListener<Metadata>
// -------------------------------------------------------------------------
@Override
public void added(Metadata metadata) {
invalidateCacheIfTimescaleDb(metadata);
}
@Override
public void updated(Metadata oldMetadata, Metadata newMetadata) {
invalidateCacheIfTimescaleDb(newMetadata);
}
@Override
public void removed(Metadata metadata) {
invalidateCacheIfTimescaleDb(metadata);
}
private void invalidateCacheIfTimescaleDb(Metadata metadata) {
if (TimescaleDBMetadataService.METADATA_NAMESPACE.equals(metadata.getUID().getNamespace())) {
String itemName = metadata.getUID().getItemName();
itemIdCache.remove(itemName);
LOGGER.debug("Invalidated item_id cache for '{}' due to metadata change", itemName);
}
}
// -------------------------------------------------------------------------
// Internal helpers
// -------------------------------------------------------------------------
private int getOrCreateItemId(Connection conn, String name, @Nullable String label) throws SQLException {
private int getOrCreateItemId(Connection conn, String name, @Nullable String label, @Nullable String value,
@Nullable String metadataJson) throws SQLException {
Integer cached = itemIdCache.get(name);
if (cached != null) {
return cached;
}
int id = TimescaleDBQuery.getOrCreateItemId(conn, name, label);
int id = TimescaleDBQuery.getOrCreateItemId(conn, name, label, value, metadataJson);
itemIdCache.put(name, id);
return id;
}
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBQuery.java
+26 -17
View File
@@ -59,7 +59,9 @@ public class TimescaleDBQuery {
// --- item_meta lookup / insert ---
private static final String SQL_SELECT_ITEM_ID = "SELECT id FROM item_meta WHERE name = ?";
private static final String SQL_INSERT_ITEM_META = "INSERT INTO item_meta (name, label) VALUES (?, ?) ON CONFLICT (name) DO UPDATE SET label = EXCLUDED.label RETURNING id";
// value = user-defined string from metadata.getValue() (stored as TEXT)
// metadata = full config map serialized as JSON (stored as JSONB via ::jsonb cast)
private static final String SQL_UPSERT_ITEM_META = "INSERT INTO item_meta (name, label, value, metadata) VALUES (?, ?, ?, ?::jsonb) ON CONFLICT (name) DO UPDATE SET label = EXCLUDED.label, value = EXCLUDED.value, metadata = EXCLUDED.metadata RETURNING id";
// --- SELECT base ---
private static final String SQL_SELECT_BASE = "SELECT time, value, string, unit FROM items WHERE item_id = ?";
@@ -100,33 +102,32 @@ public class TimescaleDBQuery {
}
/**
* Returns the item_id for the given name, or inserts a new {@code item_meta} row and returns its id.
* Returns the item_id for the given name, inserting or updating the {@code item_meta} row as needed.
*
* @param connection The JDBC connection.
* @param name The item name.
* @param label The item label (may be null; stored for informational purposes).
* @param label The item label (may be null).
* @param value The user-defined value string from {@code metadata.getValue()} (may be null), stored in
* {@code item_meta.value}.
* @param metadataJson The full config map serialized as JSON (may be null), stored in
* {@code item_meta.metadata} as JSONB.
* @return The item_id.
* @throws SQLException on any database error.
*/
public static int getOrCreateItemId(Connection connection, String name, @Nullable String label)
throws SQLException {
// Try SELECT first (fast path for known items)
try (PreparedStatement ps = connection.prepareStatement(SQL_SELECT_ITEM_ID)) {
ps.setString(1, name);
try (ResultSet rs = ps.executeQuery()) {
if (rs.next()) {
return rs.getInt(1);
}
}
}
// Not found: INSERT with ON CONFLICT DO UPDATE so concurrent calls are safe
try (PreparedStatement ps = connection.prepareStatement(SQL_INSERT_ITEM_META)) {
public static int getOrCreateItemId(Connection connection, String name, @Nullable String label,
@Nullable String value, @Nullable String metadataJson) throws SQLException {
try (PreparedStatement ps = connection.prepareStatement(SQL_UPSERT_ITEM_META)) {
ps.setString(1, name);
ps.setString(2, label);
ps.setString(3, value);
// JSONB parameter: use setObject with Types.OTHER so the driver passes it as-is
// to the ?::jsonb cast in the SQL; setString would bind it as VARCHAR which PostgreSQL
// accepts with the explicit cast, but setObject is the idiomatic approach for non-standard types.
ps.setObject(4, metadataJson, Types.OTHER);
try (ResultSet rs = ps.executeQuery()) {
if (rs.next()) {
int id = rs.getInt(1);
LOGGER.debug("Registered new item '{}' with item_id={}", name, id);
LOGGER.debug("Registered/updated item '{}' with item_id={}", name, id);
return id;
}
}
@@ -134,6 +135,14 @@ public class TimescaleDBQuery {
throw new SQLException("Failed to get or create item_meta entry for item '" + name + "'");
}
/**
* Convenience overload without value or metadata (both default to {@code null}).
*/
public static int getOrCreateItemId(Connection connection, String name, @Nullable String label)
throws SQLException {
return getOrCreateItemId(connection, name, label, null, null);
}
/**
* Queries historic items according to the given filter criteria.
*
bundles/org.openhab.persistence.timescaledb/src/main/java/org/openhab/persistence/timescaledb/internal/TimescaleDBSchema.java
+44 -1
View File
@@ -28,7 +28,8 @@ import org.slf4j.LoggerFactory;
* <p>
* Schema overview:
* <ul>
* <li>{@code item_meta} — name-to-ID lookup table for items</li>
* <li>{@code item_meta} — name-to-ID lookup table for items, stores user-defined value string and full config
* JSONB</li>
* <li>{@code items} — single hypertable for all item states</li>
* </ul>
*
@@ -44,10 +45,30 @@ public class TimescaleDBSchema {
id SERIAL PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
label TEXT,
value TEXT,
metadata JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
)
""";
/**
* Migration: adds {@code value TEXT} and {@code metadata JSONB} columns to existing installations
* that predate these columns. Single atomic ALTER TABLE acquires one lock for both columns.
* Uses {@code lock_timeout} so the service does not block startup indefinitely if another
* transaction holds a lock on {@code item_meta}. On timeout a WARNING is logged and the
* migration is retried on the next startup.
*/
private static final String SQL_MIGRATE_ADD_COLUMNS = """
DO $$ BEGIN
SET LOCAL lock_timeout = '5s';
ALTER TABLE item_meta
ADD COLUMN IF NOT EXISTS value TEXT,
ADD COLUMN IF NOT EXISTS metadata JSONB;
EXCEPTION WHEN lock_not_available THEN
RAISE WARNING 'item_meta: could not add columns within lock timeout — will retry on next startup';
END $$
""";
private static final String SQL_CREATE_ITEMS = """
CREATE TABLE IF NOT EXISTS items (
time TIMESTAMPTZ NOT NULL,
@@ -144,10 +165,14 @@ public class TimescaleDBSchema {
stmt.execute(SQL_CREATE_ITEM_META);
LOGGER.debug("Table item_meta ready");
stmt.execute(SQL_MIGRATE_ADD_COLUMNS);
stmt.execute(SQL_CREATE_ITEMS);
LOGGER.debug("Table items ready");
}
verifyItemMetaSchema(connection);
try (PreparedStatement ps = connection.prepareStatement(SQL_CREATE_HYPERTABLE)) {
ps.setString(1, chunkInterval);
ps.execute();
@@ -163,6 +188,24 @@ public class TimescaleDBSchema {
}
}
private static void verifyItemMetaSchema(Connection connection) throws SQLException {
try (PreparedStatement ps = connection.prepareStatement("SELECT column_name FROM information_schema.columns "
+ "WHERE table_name = 'item_meta' AND column_name IN ('value', 'metadata')");
ResultSet rs = ps.executeQuery()) {
var present = new java.util.HashSet<String>();
while (rs.next()) {
present.add(rs.getString(1));
}
if (!present.contains("value") || !present.contains("metadata")) {
throw new SQLException("item_meta schema migration incomplete — required columns missing: "
+ (present.contains("value") ? "" : "'value' ")
+ (present.contains("metadata") ? "" : "'metadata'")
+ "— another transaction may be holding a lock. Service will retry on next startup.");
}
}
LOGGER.debug("Columns item_meta.value and item_meta.metadata verified");
}
private static void setupCompression(Connection connection, int compressionAfterDays) throws SQLException {
try (Statement stmt = connection.createStatement()) {
stmt.execute(SQL_ENABLE_COMPRESSION);
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/BundleManifestTest.java
+2 -1
View File
@@ -57,7 +57,8 @@ class BundleManifestTest {
"org.w3c.dom.", // Java XML DOM
"org.xml.sax.", // Java XML SAX
"org.ietf.jgss.", // Java GSSAPI
"io.micrometer." // Micrometer metrics (embedded in openHAB core)
"io.micrometer.", // Micrometer metrics (embedded in openHAB core)
"com.google.gson." // Gson (embedded in openHAB core)
);
@Test
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBContainerTest.java
+263 -9
View File
@@ -500,8 +500,8 @@ class TimescaleDBContainerTest {
}
MetadataRegistry mr = mock(MetadataRegistry.class);
Metadata meta = new Metadata(new MetadataKey("timescaledb", "BoundarySensor"), "AVG",
Map.of("downsampleInterval", "2h", "retainRawDays", "0"));
Metadata meta = new Metadata(new MetadataKey("timescaledb", "BoundarySensor"), "sensor.boundary",
Map.of("aggregation", "AVG", "downsampleInterval", "2h", "retainRawDays", "0"));
when(mr.get(new MetadataKey("timescaledb", "BoundarySensor"))).thenReturn(meta);
when(mr.getAll()).thenAnswer(inv -> List.of(meta));
@@ -554,8 +554,8 @@ class TimescaleDBContainerTest {
// Configure metadata for downsampling with retainRawDays=0
MetadataRegistry metadataRegistry = mock(MetadataRegistry.class);
Metadata meta = new Metadata(new MetadataKey("timescaledb", "DownsampleSensor"), "AVG",
Map.of("downsampleInterval", "2h", "retainRawDays", "0"));
Metadata meta = new Metadata(new MetadataKey("timescaledb", "DownsampleSensor"), "sensor.downsample",
Map.of("aggregation", "AVG", "downsampleInterval", "2h", "retainRawDays", "0"));
when(metadataRegistry.get(new MetadataKey("timescaledb", "DownsampleSensor"))).thenReturn(meta);
when(metadataRegistry.getAll()).thenAnswer(inv -> List.of(meta));
@@ -610,8 +610,8 @@ class TimescaleDBContainerTest {
MetadataRegistry mr = mock(MetadataRegistry.class);
// retentionDays=30 → the 60-day-old row should be deleted
Metadata meta = new Metadata(new MetadataKey("timescaledb", "RetentionSensor"), "AVG",
Map.of("downsampleInterval", "1h", "retainRawDays", "0", "retentionDays", "30"));
Metadata meta = new Metadata(new MetadataKey("timescaledb", "RetentionSensor"), "sensor.retention",
Map.of("aggregation", "AVG", "downsampleInterval", "1h", "retainRawDays", "0", "retentionDays", "30"));
when(mr.get(new MetadataKey("timescaledb", "RetentionSensor"))).thenReturn(meta);
when(mr.getAll()).thenAnswer(inv -> List.of(meta));
@@ -700,7 +700,7 @@ class TimescaleDBContainerTest {
void serviceActivateInitializesschemaandschedulesjob() throws Exception {
MetadataRegistry mr = mock(MetadataRegistry.class);
when(mr.getAll()).thenReturn(Collections.emptyList());
TimescaleDBPersistenceService service = new TimescaleDBPersistenceService(mock(ItemRegistry.class),
TimescaleDBPersistenceService service = new TimescaleDBPersistenceService(mock(ItemRegistry.class), mr,
new TimescaleDBMetadataService(mr));
service.activate(Map.of("url", DB.getJdbcUrl(), "user", DB.getUsername(), "password", DB.getPassword()));
@@ -727,7 +727,7 @@ class TimescaleDBContainerTest {
NumberItem item = new NumberItem("ServiceSensor");
when(ir.getItem("ServiceSensor")).thenReturn(item);
TimescaleDBPersistenceService service = new TimescaleDBPersistenceService(ir,
TimescaleDBPersistenceService service = new TimescaleDBPersistenceService(ir, mr,
new TimescaleDBMetadataService(mr));
service.activate(Map.of("url", DB.getJdbcUrl(), "user", DB.getUsername(), "password", DB.getPassword()));
@@ -754,7 +754,7 @@ class TimescaleDBContainerTest {
void serviceDeactivateCancelsscheduledfuture() throws Exception {
MetadataRegistry mr = mock(MetadataRegistry.class);
when(mr.getAll()).thenReturn(Collections.emptyList());
TimescaleDBPersistenceService service = new TimescaleDBPersistenceService(mock(ItemRegistry.class),
TimescaleDBPersistenceService service = new TimescaleDBPersistenceService(mock(ItemRegistry.class), mr,
new TimescaleDBMetadataService(mr));
service.activate(Map.of("url", DB.getJdbcUrl(), "user", DB.getUsername(), "password", DB.getPassword()));
@@ -772,6 +772,260 @@ class TimescaleDBContainerTest {
// Helpers
// ------------------------------------------------------------------
// ------------------------------------------------------------------
// item_meta.value + metadata JSONB (integration)
// ------------------------------------------------------------------
@Test
@Order(80)
void valueStringIsStoredInItemMetaValueColumn() throws SQLException {
try (Connection conn = dataSource.getConnection()) {
TimescaleDBQuery.getOrCreateItemId(conn, "ValueSensor", "label", "sensor.temperature", null);
}
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn.prepareStatement("SELECT value FROM item_meta WHERE name = 'ValueSensor'");
ResultSet rs = ps.executeQuery()) {
assertTrue(rs.next());
assertEquals("sensor.temperature", rs.getString(1));
}
}
@Test
@Order(81)
void valueStringIsUpdatedOnUpsert() throws SQLException {
try (Connection conn = dataSource.getConnection()) {
TimescaleDBQuery.getOrCreateItemId(conn, "UpdatableSensor", "label", "old.value", null);
TimescaleDBQuery.getOrCreateItemId(conn, "UpdatableSensor", "label", "new.value", null);
}
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn
.prepareStatement("SELECT value FROM item_meta WHERE name = 'UpdatableSensor'");
ResultSet rs = ps.executeQuery()) {
assertTrue(rs.next());
assertEquals("new.value", rs.getString(1));
}
}
@Test
@Order(82)
void nullValueAndMetadataStoreNulls() throws SQLException {
try (Connection conn = dataSource.getConnection()) {
TimescaleDBQuery.getOrCreateItemId(conn, "NoMetaSensor", null);
}
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn
.prepareStatement("SELECT value, metadata FROM item_meta WHERE name = 'NoMetaSensor'");
ResultSet rs = ps.executeQuery()) {
assertTrue(rs.next());
assertNull(rs.getString(1), "value must be NULL when not provided");
assertNull(rs.getString(2), "metadata must be NULL when not provided");
}
}
@Test
@Order(83)
void metadataJsonbIsStoredAndQueryableViaJsonbOperators() throws SQLException {
String json = "{\"aggregation\":\"AVG\",\"location\":\"kitchen\"}";
try (Connection conn = dataSource.getConnection()) {
TimescaleDBQuery.getOrCreateItemId(conn, "JsonSensor", null, "sensor.temp", json);
}
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn
.prepareStatement("SELECT name FROM item_meta WHERE metadata->>'location' = 'kitchen'");
ResultSet rs = ps.executeQuery()) {
assertTrue(rs.next(), "JSONB operator ->> must work for filtering");
assertEquals("JsonSensor", rs.getString(1));
}
}
@Test
@Order(84)
void schemaMigrationAddsValueTextAndMetadataJsonbToProductionSchema() throws SQLException {
// Simulate the real production schema: item_meta without value/metadata columns
try (Connection conn = dataSource.getConnection(); var stmt = conn.createStatement()) {
stmt.execute("DROP TABLE IF EXISTS items CASCADE");
stmt.execute("DROP TABLE IF EXISTS item_meta CASCADE");
stmt.execute("CREATE TABLE item_meta (id SERIAL PRIMARY KEY, name TEXT NOT NULL UNIQUE, "
+ "label TEXT, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW())");
}
try (Connection conn = dataSource.getConnection()) {
TimescaleDBSchema.initialize(conn, "7 days", 0, 0);
}
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn
.prepareStatement("SELECT column_name, data_type " + "FROM information_schema.columns "
+ "WHERE table_name = 'item_meta' AND column_name IN ('value', 'metadata') "
+ "ORDER BY column_name");
ResultSet rs = ps.executeQuery()) {
assertTrue(rs.next(), "metadata column must have been added");
assertEquals("metadata", rs.getString("column_name"));
assertEquals("jsonb", rs.getString("data_type"), "metadata must be JSONB");
assertTrue(rs.next(), "value column must have been added");
assertEquals("value", rs.getString("column_name"));
assertEquals("text", rs.getString("data_type"), "value must be TEXT");
}
}
@Test
@Order(85)
void schemaMigrationDoesNotBlockWhenAnotherTransactionLocksItemMeta() throws Exception {
// Simulate the production schema without value/metadata columns
try (Connection conn = dataSource.getConnection(); var stmt = conn.createStatement()) {
stmt.execute("DROP TABLE IF EXISTS items CASCADE");
stmt.execute("DROP TABLE IF EXISTS item_meta CASCADE");
stmt.execute("CREATE TABLE item_meta (id SERIAL PRIMARY KEY, name TEXT NOT NULL UNIQUE, "
+ "label TEXT, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW())");
}
// Open a transaction that holds a lock on item_meta — simulates a long-running query
Connection blockingConn = dataSource.getConnection();
blockingConn.setAutoCommit(false);
try (var stmt = blockingConn.createStatement()) {
stmt.execute("SELECT * FROM item_meta FOR SHARE");
}
// initialize() must fail quickly (lock_timeout) instead of blocking indefinitely.
// It must throw a SQLException to prevent the service from starting in a broken state.
long start = System.currentTimeMillis();
try (Connection conn = dataSource.getConnection()) {
assertThrows(SQLException.class, () -> TimescaleDBSchema.initialize(conn, "7 days", 0, 0),
"initialize() must throw when migration cannot complete — service must not start with missing columns");
} finally {
blockingConn.rollback();
blockingConn.close();
}
long elapsed = System.currentTimeMillis() - start;
assertTrue(elapsed < 15_000,
"Schema initialization must fail quickly, not block indefinitely — took " + elapsed + "ms");
}
// ------------------------------------------------------------------
// Migration end-to-end: existing data must survive schema changes
// ------------------------------------------------------------------
/**
* Simulates the production upgrade path from the earliest schema
* (item_meta with only id/name/label/created_at, no value/metadata columns)
* to the current schema. Existing rows in item_meta and their linked
* items entries must survive the migration intact.
*/
@Test
@Order(86)
void schemaMigrationPreservesExistingRowsWhenUpgradingFromOriginalSchema() throws SQLException {
// @BeforeEach already created the full current schema — insert real data first,
// then strip the new columns to simulate a pre-migration production state.
int existingId;
try (Connection conn = dataSource.getConnection(); var stmt = conn.createStatement()) {
// Insert an existing item via the current API
existingId = TimescaleDBQuery.getOrCreateItemId(conn, "LegacySensor", "Living Room Temp");
// Insert a measurement for that item
TimescaleDBQuery.insert(conn, existingId, ZonedDateTime.now().minusHours(1),
new TimescaleDBMapper.Row(21.5, null, null));
// Now simulate the old schema: drop the columns that did not exist originally
stmt.execute("ALTER TABLE item_meta DROP COLUMN IF EXISTS value");
stmt.execute("ALTER TABLE item_meta DROP COLUMN IF EXISTS metadata");
}
// Run migration — must succeed without error
try (Connection conn = dataSource.getConnection()) {
TimescaleDBSchema.initialize(conn, "7 days", 0, 0);
}
// Verify: item_meta row survived with correct name and label
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn
.prepareStatement("SELECT id, name, label FROM item_meta WHERE name = 'LegacySensor'");
ResultSet rs = ps.executeQuery()) {
assertTrue(rs.next(), "item_meta row for LegacySensor must survive migration");
assertEquals(existingId, rs.getInt("id"), "item_meta.id must not change during migration");
assertEquals("Living Room Temp", rs.getString("label"));
}
// Verify: items measurement still referenced by its original item_meta.id
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn.prepareStatement("SELECT COUNT(*) FROM items WHERE item_id = ?")) {
ps.setInt(1, existingId);
ResultSet rs = ps.executeQuery();
assertTrue(rs.next());
assertEquals(1, rs.getInt(1), "items row must survive migration — FK reference must remain valid");
}
// Verify: new columns were added with NULL for old rows
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn
.prepareStatement("SELECT value, metadata FROM item_meta WHERE name = 'LegacySensor'");
ResultSet rs = ps.executeQuery()) {
assertTrue(rs.next());
assertNull(rs.getString("value"), "value must be NULL for rows that predate the migration");
assertNull(rs.getString("metadata"), "metadata must be NULL for rows that predate the migration");
}
// Verify: service is fully operational after migration — new store + query round-trip works
try (Connection conn = dataSource.getConnection()) {
int id = TimescaleDBQuery.getOrCreateItemId(conn, "LegacySensor", "Living Room Temp");
assertEquals(existingId, id, "getOrCreateItemId must return the existing row, not create a duplicate");
TimescaleDBQuery.insert(conn, id, ZonedDateTime.now(), new TimescaleDBMapper.Row(22.0, null, null));
FilterCriteria filter = new FilterCriteria();
filter.setItemName("LegacySensor");
NumberItem item = new NumberItem("LegacySensor");
List<HistoricItem> results = TimescaleDBQuery.query(conn, item, id, filter);
assertFalse(results.isEmpty(), "Query must return results after migration");
}
}
/**
* initialize() must be idempotent: running it a second time on an already-migrated schema
* must not throw, not duplicate rows, and not corrupt existing data.
*/
@Test
@Order(87)
void schemaInitializeIsIdempotent() throws SQLException {
// @BeforeEach already ran initialize() once — insert data
int existingId;
try (Connection conn = dataSource.getConnection()) {
existingId = TimescaleDBQuery.getOrCreateItemId(conn, "IdempotentSensor", "Idempotency Test", "sensor.test",
null);
TimescaleDBQuery.insert(conn, existingId, ZonedDateTime.now().minusMinutes(5),
new TimescaleDBMapper.Row(42.0, null, null));
}
// Run initialize() a second time — must not throw
assertDoesNotThrow(() -> {
try (Connection conn = dataSource.getConnection()) {
TimescaleDBSchema.initialize(conn, "7 days", 0, 0);
}
}, "initialize() must be idempotent — running it twice must not throw");
// item_meta row must not be duplicated
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn
.prepareStatement("SELECT COUNT(*) FROM item_meta WHERE name = 'IdempotentSensor'");
ResultSet rs = ps.executeQuery()) {
assertTrue(rs.next());
assertEquals(1, rs.getInt(1), "Second initialize() must not duplicate item_meta rows");
}
// items data must still be intact
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn.prepareStatement("SELECT COUNT(*) FROM items WHERE item_id = ?")) {
ps.setInt(1, existingId);
ResultSet rs = ps.executeQuery();
assertTrue(rs.next());
assertEquals(1, rs.getInt(1), "items data must survive second initialize() call");
}
}
@SafeVarargs
private void storeAndVerify(String itemName, org.openhab.core.items.Item item, State state,
java.util.function.Consumer<State>... assertions) throws SQLException {
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBDownsampleJobTest.java
+16 -13
View File
@@ -80,7 +80,8 @@ class TimescaleDBDownsampleJobTest {
@Test
void runSingleitemExecutesinsertanddeleteintransaction() throws SQLException {
stubMetadata("SensorA", "AVG", Map.of("downsampleInterval", "1h", "retainRawDays", "5"));
stubMetadata("SensorA", "sensor.a",
Map.of("aggregation", "AVG", "downsampleInterval", "1h", "retainRawDays", "5"));
stubItemNames(List.of("SensorA"));
stubItemIdInDb(connection, "SensorA", 42);
@@ -99,8 +100,8 @@ class TimescaleDBDownsampleJobTest {
@Test
void runWithretentiondaysExecutesthreestatements() throws SQLException {
stubMetadata("SensorA", "MAX",
Map.of("downsampleInterval", "15m", "retainRawDays", "3", "retentionDays", "90"));
stubMetadata("SensorA", "sensor.a",
Map.of("aggregation", "MAX", "downsampleInterval", "15m", "retainRawDays", "3", "retentionDays", "90"));
stubItemNames(List.of("SensorA"));
stubItemIdInDb(connection, "SensorA", 7);
@@ -113,7 +114,7 @@ class TimescaleDBDownsampleJobTest {
@Test
void runItemnotInDbSkipsWithoutDml() throws SQLException {
stubMetadata("UnknownItem", "AVG", Map.of("downsampleInterval", "1h"));
stubMetadata("UnknownItem", "sensor.u", Map.of("aggregation", "AVG", "downsampleInterval", "1h"));
stubItemNames(List.of("UnknownItem"));
// No stubItemIdInDb → SELECT returns empty (default setUp)
@@ -127,8 +128,8 @@ class TimescaleDBDownsampleJobTest {
@Test
void runSqlfailureforoneitemDoesnotabortotheritems() throws SQLException {
stubMetadata("SensorA", "AVG", Map.of("downsampleInterval", "1h"));
stubMetadata("SensorB", "SUM", Map.of("downsampleInterval", "1d"));
stubMetadata("SensorA", "sensor.a", Map.of("aggregation", "AVG", "downsampleInterval", "1h"));
stubMetadata("SensorB", "sensor.b", Map.of("aggregation", "SUM", "downsampleInterval", "1d"));
stubItemNames(List.of("SensorA", "SensorB"));
// First connection: SELECT succeeds (item found), INSERT fails
@@ -246,7 +247,8 @@ class TimescaleDBDownsampleJobTest {
@Test
void runInsertsqlcontainscorrectintervalandfunction() throws SQLException {
stubMetadata("Sensor", "MIN", Map.of("downsampleInterval", "6h", "retainRawDays", "3"));
stubMetadata("Sensor", "sensor.s",
Map.of("aggregation", "MIN", "downsampleInterval", "6h", "retainRawDays", "3"));
stubItemNames(List.of("Sensor"));
PreparedStatement selectPs = mock(PreparedStatement.class);
@@ -277,7 +279,8 @@ class TimescaleDBDownsampleJobTest {
@Test
void runDeletesqlreferencesretainrawdays() throws SQLException {
stubMetadata("Sensor", "AVG", Map.of("downsampleInterval", "1h", "retainRawDays", "7"));
stubMetadata("Sensor", "sensor.s",
Map.of("aggregation", "AVG", "downsampleInterval", "1h", "retainRawDays", "7"));
stubItemNames(List.of("Sensor"));
PreparedStatement selectPs = mock(PreparedStatement.class);
@@ -307,7 +310,7 @@ class TimescaleDBDownsampleJobTest {
@Test
void runRollbackonsqlerror() throws SQLException {
stubMetadata("SensorA", "AVG", Map.of("downsampleInterval", "1h"));
stubMetadata("SensorA", "sensor.a", Map.of("aggregation", "AVG", "downsampleInterval", "1h"));
stubItemNames(List.of("SensorA"));
stubItemIdInDb(connection, "SensorA", 1);
@@ -343,14 +346,14 @@ class TimescaleDBDownsampleJobTest {
}
private void stubItemNames(List<String> names) {
var metaList = names.stream()
.map(n -> new Metadata(new MetadataKey("timescaledb", n), "AVG", Map.of("downsampleInterval", "1h")))
.toList();
var metaList = names.stream().map(n -> new Metadata(new MetadataKey("timescaledb", n),
"sensor." + n.toLowerCase(), Map.of("aggregation", "AVG", "downsampleInterval", "1h"))).toList();
when(registry.getAll()).thenAnswer(inv -> metaList);
for (String name : names) {
MetadataKey key = new MetadataKey("timescaledb", name);
if (registry.get(key) == null) {
when(registry.get(key)).thenReturn(new Metadata(key, "AVG", Map.of("downsampleInterval", "1h")));
when(registry.get(key)).thenReturn(new Metadata(key, "sensor." + name.toLowerCase(),
Map.of("aggregation", "AVG", "downsampleInterval", "1h")));
}
}
}
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBMetadataServiceTest.java
+117 -25
View File
@@ -49,18 +49,19 @@ class TimescaleDBMetadataServiceTest {
}
// ------------------------------------------------------------------
// getDownsampleConfig — happy paths
// getDownsampleConfig — happy paths (aggregation now in config map)
// ------------------------------------------------------------------
@ParameterizedTest
@CsvSource({ "AVG,1h,1 hour", "MAX,15m,15 minutes", "MIN,1d,1 day", "SUM,30m,30 minutes" })
void getDownsampleConfigValidfunctionandinterval(String function, String interval, String expectedSql) {
stubMetadata("MySensor", function, Map.of("downsampleInterval", interval));
void getDownsampleConfigValidaggregationandinterval(String aggregation, String interval, String expectedSql) {
stubMetadata("MySensor", "sensor.temperature",
Map.of("aggregation", aggregation, "downsampleInterval", interval));
var config = service.getDownsampleConfig("MySensor");
assertTrue(config.isPresent());
assertEquals(AggregationFunction.valueOf(function), config.get().function());
assertEquals(AggregationFunction.valueOf(aggregation), config.get().function());
assertEquals(expectedSql, config.get().sqlInterval());
assertEquals(5, config.get().retainRawDays()); // default
assertEquals(0, config.get().retentionDays()); // default
@@ -68,8 +69,8 @@ class TimescaleDBMetadataServiceTest {
@Test
void getDownsampleConfigCustomretainrawandretentiondays() {
stubMetadata("MySensor", "AVG",
Map.of("downsampleInterval", "1h", "retainRawDays", "7", "retentionDays", "365"));
stubMetadata("MySensor", "sensor.temperature",
Map.of("aggregation", "AVG", "downsampleInterval", "1h", "retainRawDays", "7", "retentionDays", "365"));
var config = service.getDownsampleConfig("MySensor").orElseThrow();
@@ -80,7 +81,8 @@ class TimescaleDBMetadataServiceTest {
@Test
void getDownsampleConfigAllsupportedintervals() {
for (Map.Entry<String, String> entry : DownsampleConfig.INTERVAL_MAP.entrySet()) {
stubMetadata("Item_" + entry.getKey(), "AVG", Map.of("downsampleInterval", entry.getKey()));
stubMetadata("Item_" + entry.getKey(), "my.sensor",
Map.of("aggregation", "AVG", "downsampleInterval", entry.getKey()));
var config = service.getDownsampleConfig("Item_" + entry.getKey());
assertTrue(config.isPresent(), "Should parse interval: " + entry.getKey());
assertEquals(entry.getValue(), config.get().sqlInterval());
@@ -88,7 +90,7 @@ class TimescaleDBMetadataServiceTest {
}
// ------------------------------------------------------------------
// getDownsampleConfig — no / empty metadata
// getDownsampleConfig — no / empty aggregation
// ------------------------------------------------------------------
@Test
@@ -99,9 +101,9 @@ class TimescaleDBMetadataServiceTest {
}
@Test
void getDownsampleConfigBlankfunctionWithretentiondaysReturnsretentiononlyconfig() {
// Blank value + retentionDays → retention-only config (no downsampling)
stubMetadata("MySensor", " ", Map.of("retentionDays", "30"));
void getDownsampleConfigNoAggregationKeyWithRetentiondaysReturnsRetentiononlyconfig() {
// No aggregation key + retentionDays → retention-only config
stubMetadata("MySensor", "my.sensor", Map.of("retentionDays", "30"));
Optional<DownsampleConfig> result = service.getDownsampleConfig("MySensor");
assertTrue(result.isPresent());
@@ -112,13 +114,23 @@ class TimescaleDBMetadataServiceTest {
}
@Test
void getDownsampleConfigBlankfunctionWithoutretentiondaysReturnsempty() {
// Blank value + no retentionDays → nothing to do, skip item
stubMetadata("MySensor", " ", Map.of());
void getDownsampleConfigNoAggregationKeyWithoutRetentiondaysReturnsempty() {
// No aggregation key + no retentionDays → nothing to do
stubMetadata("MySensor", "my.sensor", Map.of());
assertTrue(service.getDownsampleConfig("MySensor").isEmpty());
}
@Test
void getDownsampleConfigBlankValueFieldDoesNotAffectAggregationParsing() {
// getValue() is now user-defined label — a blank value must not affect aggregation parsing
stubMetadata("MySensor", " ", Map.of("aggregation", "AVG", "downsampleInterval", "1h"));
Optional<DownsampleConfig> result = service.getDownsampleConfig("MySensor");
assertTrue(result.isPresent());
assertEquals(AggregationFunction.AVG, result.get().function());
}
// ------------------------------------------------------------------
// getDownsampleConfig — invalid / unsupported values
// ------------------------------------------------------------------
@@ -127,33 +139,32 @@ class TimescaleDBMetadataServiceTest {
@ValueSource(strings = { "2h30m", "3m", "1w", "invalid", "" })
void getDownsampleConfigInvalidintervalReturnsempty(String badInterval) {
if (badInterval.isBlank()) {
// handled by the missing-interval branch
stubMetadata("MySensor", "AVG", Map.of());
stubMetadata("MySensor", "s", Map.of("aggregation", "AVG"));
} else {
stubMetadata("MySensor", "AVG", Map.of("downsampleInterval", badInterval));
stubMetadata("MySensor", "s", Map.of("aggregation", "AVG", "downsampleInterval", badInterval));
}
assertTrue(service.getDownsampleConfig("MySensor").isEmpty());
}
@Test
void getDownsampleConfigInvalidfunctionReturnsempty() {
stubMetadata("MySensor", "MEDIAN", Map.of("downsampleInterval", "1h"));
void getDownsampleConfigInvalidaggregationReturnsempty() {
stubMetadata("MySensor", "s", Map.of("aggregation", "MEDIAN", "downsampleInterval", "1h"));
assertTrue(service.getDownsampleConfig("MySensor").isEmpty());
}
@Test
void getDownsampleConfigMissingintervalReturnsempty() {
// Function present but no interval → cannot downsample
stubMetadata("MySensor", "AVG", Map.of());
stubMetadata("MySensor", "s", Map.of("aggregation", "AVG"));
assertTrue(service.getDownsampleConfig("MySensor").isEmpty());
}
@Test
void getDownsampleConfigInvalidretainrawdaysUsesdefault() {
stubMetadata("MySensor", "AVG", Map.of("downsampleInterval", "1h", "retainRawDays", "not-a-number"));
stubMetadata("MySensor", "s",
Map.of("aggregation", "AVG", "downsampleInterval", "1h", "retainRawDays", "not-a-number"));
var config = service.getDownsampleConfig("MySensor").orElseThrow();
assertEquals(5, config.retainRawDays()); // falls back to default
@@ -181,11 +192,12 @@ class TimescaleDBMetadataServiceTest {
@Test
void getConfiguredItemNamesReturnsallTimescaledbitemsRegardlessofvalue() {
Metadata withFunction = metadata("SensorA", "AVG", Map.of("downsampleInterval", "1h"));
Metadata retentionOnly = metadata("SensorB", " ", Map.of("retentionDays", "30"));
Metadata withAggregation = metadata("SensorA", "sensor.a",
Map.of("aggregation", "AVG", "downsampleInterval", "1h"));
Metadata retentionOnly = metadata("SensorB", "sensor.b", Map.of("retentionDays", "30"));
Metadata otherNamespace = new Metadata(new MetadataKey("influxdb", "SensorC"), "some", Map.of());
when(registry.getAll()).thenAnswer(inv -> List.of(withFunction, retentionOnly, otherNamespace));
when(registry.getAll()).thenAnswer(inv -> List.of(withAggregation, retentionOnly, otherNamespace));
List<String> names = service.getConfiguredItemNames();
@@ -201,6 +213,86 @@ class TimescaleDBMetadataServiceTest {
assertTrue(service.getConfiguredItemNames().isEmpty());
}
// ------------------------------------------------------------------
// getMetadataValueString
// ------------------------------------------------------------------
@Test
void getMetadataValueStringNometadataReturnsNull() {
when(registry.get(new MetadataKey("timescaledb", "Unknown"))).thenReturn(null);
assertNull(service.getMetadataValueString("Unknown"));
}
@Test
void getMetadataValueStringReturnsValueField() {
stubMetadata("MySensor", "sensor.temperature", Map.of("aggregation", "AVG", "downsampleInterval", "1h"));
assertEquals("sensor.temperature", service.getMetadataValueString("MySensor"));
}
@Test
void getMetadataValueStringBlankValueReturnsNull() {
stubMetadata("MySensor", " ", Map.of("retentionDays", "30"));
assertNull(service.getMetadataValueString("MySensor"),
"Blank getValue() must be treated as absent and return null");
}
@Test
void getMetadataValueStringDoesNotUseConfigKeys() {
// The value field is getValue(), not any config key — aggregation must NOT appear here
stubMetadata("MySensor", "sensor.temperature", Map.of("aggregation", "AVG", "downsampleInterval", "1h"));
assertEquals("sensor.temperature", service.getMetadataValueString("MySensor"));
assertNotEquals("AVG", service.getMetadataValueString("MySensor"),
"getMetadataValueString must return getValue(), not the aggregation config key");
}
// ------------------------------------------------------------------
// getMetadataConfigJson
// ------------------------------------------------------------------
@Test
void getMetadataConfigJsonNometadataReturnsNull() {
when(registry.get(new MetadataKey("timescaledb", "Unknown"))).thenReturn(null);
assertNull(service.getMetadataConfigJson("Unknown"));
}
@Test
void getMetadataConfigJsonEmptyConfigReturnsNull() {
stubMetadata("MySensor", "sensor.temperature", Map.of());
assertNull(service.getMetadataConfigJson("MySensor"));
}
@Test
void getMetadataConfigJsonReturnsSerializedMap() {
stubMetadata("MySensor", "sensor.temperature", Map.of("aggregation", "AVG", "downsampleInterval", "1h"));
String json = service.getMetadataConfigJson("MySensor");
assertNotNull(json);
assertTrue(json.startsWith("{"), "Must be a JSON object");
assertTrue(json.contains("\"aggregation\""), "Must contain aggregation key");
assertTrue(json.contains("\"AVG\""), "Must contain aggregation value");
assertTrue(json.contains("\"downsampleInterval\""), "Must contain downsampleInterval key");
}
@Test
void getMetadataConfigJsonIncludesAllConfigKeys() {
// All config keys must be stored — no filtering
stubMetadata("MySensor", "sensor.temperature", Map.of("aggregation", "AVG", "downsampleInterval", "1h",
"retainRawDays", "5", "retentionDays", "365", "location", "living_room", "kind", "sensor"));
String json = service.getMetadataConfigJson("MySensor");
assertNotNull(json);
assertTrue(json.contains("\"location\""), "User-defined tag 'location' must be stored");
assertTrue(json.contains("\"kind\""), "User-defined tag 'kind' must be stored");
assertTrue(json.contains("\"retainRawDays\""), "Reserved key retainRawDays must be stored");
assertTrue(json.contains("\"retentionDays\""), "Reserved key retentionDays must be stored");
}
// ------------------------------------------------------------------
// Helpers
// ------------------------------------------------------------------
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBPersistenceServiceTest.java
+155 -25
View File
@@ -67,7 +67,7 @@ class TimescaleDBPersistenceServiceTest {
private final HikariDataSource injectedDs;
TestableService(ItemRegistry ir, MetadataRegistry mr, HikariDataSource ds) {
super(ir, new TimescaleDBMetadataService(mr));
super(ir, mr, new TimescaleDBMetadataService(mr));
this.injectedDs = ds;
}
@@ -140,7 +140,7 @@ class TimescaleDBPersistenceServiceTest {
@Test
void storeNormalstateSendsinsert() throws Exception {
// item_id cache is empty → getOrCreateItemId will run SELECT then INSERT
// item_id cache is empty → getOrCreateItemId will run UPSERT
stubItemIdLookup(7);
var item = new NumberItem("Sensor1");
@@ -164,16 +164,67 @@ class TimescaleDBPersistenceServiceTest {
var item = new NumberItem("RealName");
// Capture which PreparedStatements get setString(1, "AliasName")
PreparedStatement selectPs = mock(PreparedStatement.class);
ResultSet selectRs = mock(ResultSet.class);
when(selectRs.next()).thenReturn(false);
when(selectPs.executeQuery()).thenReturn(selectRs);
when(connection.prepareStatement(contains("SELECT id FROM item_meta"))).thenReturn(selectPs);
PreparedStatement upsertPs = mock(PreparedStatement.class);
ResultSet upsertRs = mock(ResultSet.class);
when(upsertRs.next()).thenReturn(true);
when(upsertRs.getInt(1)).thenReturn(3);
when(upsertPs.executeQuery()).thenReturn(upsertRs);
when(connection.prepareStatement(contains("INSERT INTO item_meta"))).thenReturn(upsertPs);
service.store(item, ZonedDateTime.now(), new DecimalType(1.0), "AliasName");
// The item_id lookup SELECT must be called with the alias
verify(selectPs, atLeastOnce()).setString(eq(1), eq("AliasName"));
// The item_id UPSERT must be called with the alias as parameter 1
verify(upsertPs, atLeastOnce()).setString(eq(1), eq("AliasName"));
}
@Test
void storeValueStringIsPassedToItemMetaUpsert() throws Exception {
var upsertPs = mock(PreparedStatement.class);
var upsertRs = mock(ResultSet.class);
var insertItemPs = mock(PreparedStatement.class);
when(upsertRs.next()).thenReturn(true);
when(upsertRs.getInt(1)).thenReturn(10);
when(upsertPs.executeQuery()).thenReturn(upsertRs);
when(insertItemPs.executeUpdate()).thenReturn(1);
when(connection.prepareStatement(contains("INSERT INTO item_meta"))).thenReturn(upsertPs);
when(connection.prepareStatement(contains("INSERT INTO items"))).thenReturn(insertItemPs);
// New format: value = "my.sensor" (getValue()), aggregation in config map
var metaKey = new org.openhab.core.items.MetadataKey("timescaledb", "Sensor1");
var meta = new org.openhab.core.items.Metadata(metaKey, "my.sensor",
Map.of("aggregation", "AVG", "downsampleInterval", "1h"));
when(metadataRegistry.get(metaKey)).thenReturn(meta);
service.store(new NumberItem("Sensor1"), ZonedDateTime.now(), new DecimalType(1.0), null);
// Parameter 3 = value string (getText from getValue())
verify(upsertPs).setString(3, "my.sensor");
}
@Test
void storeMetadataConfigJsonIsPassedToItemMetaUpsert() throws Exception {
var upsertPs = mock(PreparedStatement.class);
var upsertRs = mock(ResultSet.class);
var insertItemPs = mock(PreparedStatement.class);
when(upsertRs.next()).thenReturn(true);
when(upsertRs.getInt(1)).thenReturn(11);
when(upsertPs.executeQuery()).thenReturn(upsertRs);
when(insertItemPs.executeUpdate()).thenReturn(1);
when(connection.prepareStatement(contains("INSERT INTO item_meta"))).thenReturn(upsertPs);
when(connection.prepareStatement(contains("INSERT INTO items"))).thenReturn(insertItemPs);
var metaKey = new org.openhab.core.items.MetadataKey("timescaledb", "Sensor1");
var meta = new org.openhab.core.items.Metadata(metaKey, "my.sensor",
Map.of("aggregation", "AVG", "location", "kitchen"));
when(metadataRegistry.get(metaKey)).thenReturn(meta);
service.store(new NumberItem("Sensor1"), ZonedDateTime.now(), new DecimalType(1.0), null);
// Parameter 4 = metadata JSONB — must use setObject with Types.OTHER and contain a JSON string
verify(upsertPs).setObject(eq(4), argThat(arg -> {
String s = String.valueOf(arg);
return s.contains("aggregation") && s.contains("AVG") && s.contains("location");
}), eq(java.sql.Types.OTHER));
}
// ------------------------------------------------------------------
@@ -310,7 +361,7 @@ class TimescaleDBPersistenceServiceTest {
@Test
void activateMissingurlDatasourceremainsnull() throws Exception {
var realService = new TimescaleDBPersistenceService(mock(ItemRegistry.class),
var realService = new TimescaleDBPersistenceService(mock(ItemRegistry.class), mock(MetadataRegistry.class),
new TimescaleDBMetadataService(mock(MetadataRegistry.class)));
realService.activate(Map.of()); // no 'url' key
@@ -322,7 +373,7 @@ class TimescaleDBPersistenceServiceTest {
@Test
void activateInvalidurlDatasourceremainsnull() throws Exception {
var realService = new TimescaleDBPersistenceService(mock(ItemRegistry.class),
var realService = new TimescaleDBPersistenceService(mock(ItemRegistry.class), mock(MetadataRegistry.class),
new TimescaleDBMetadataService(mock(MetadataRegistry.class)));
// Unreachable host; short timeout so the test does not block long
realService.activate(
@@ -403,7 +454,7 @@ class TimescaleDBPersistenceServiceTest {
void runDownsampleNowReturnsFalseWhenNotActivated() {
// A fresh service with no activate() call has no job instance
TimescaleDBPersistenceService fresh = new TimescaleDBPersistenceService(mock(ItemRegistry.class),
new TimescaleDBMetadataService(mock(MetadataRegistry.class)));
mock(MetadataRegistry.class), new TimescaleDBMetadataService(mock(MetadataRegistry.class)));
assertFalse(fresh.runDownsampleNow(), "runDownsampleNow() must return false before activate()");
}
@@ -426,7 +477,7 @@ class TimescaleDBPersistenceServiceTest {
@Test
void consoleCommandDownsamplePrintsNotActiveWhenServiceInactive() {
TimescaleDBPersistenceService fresh = new TimescaleDBPersistenceService(mock(ItemRegistry.class),
new TimescaleDBMetadataService(mock(MetadataRegistry.class)));
mock(MetadataRegistry.class), new TimescaleDBMetadataService(mock(MetadataRegistry.class)));
var console = mock(org.openhab.core.io.console.Console.class);
new TimescaleDBConsoleCommandExtension(fresh).execute(new String[] { "downsample" }, console);
verify(console).println(contains("not active"));
@@ -440,29 +491,108 @@ class TimescaleDBPersistenceServiceTest {
verify(console, atLeastOnce()).printUsage(anyString());
}
// ------------------------------------------------------------------
// Metadata cache invalidation (RegistryChangeListener)
// ------------------------------------------------------------------
@Test
void metadataAddedInvalidatesCacheSoNextStoreRunsUpsert() throws Exception {
// Populate the cache via a first store
stubItemIdLookup(5);
service.store(new NumberItem("Sensor1"), ZonedDateTime.now(), new DecimalType(1.0), null);
// Signal metadata added → cache must be invalidated
var metaKey = new org.openhab.core.items.MetadataKey("timescaledb", "Sensor1");
var meta = new org.openhab.core.items.Metadata(metaKey, "new.label", Map.of("aggregation", "MAX"));
service.added(meta);
// Stub a fresh upsert for the second store
stubItemIdLookup(5);
reset(dataSource);
when(dataSource.getConnection()).thenReturn(connection);
service.store(new NumberItem("Sensor1"), ZonedDateTime.now(), new DecimalType(2.0), null);
// The upsert must have run again (connection obtained)
verify(dataSource, atLeastOnce()).getConnection();
verify(connection, atLeastOnce()).prepareStatement(contains("INSERT INTO item_meta"));
}
@Test
void metadataUpdatedInvalidatesCache() throws Exception {
stubItemIdLookup(6);
service.store(new NumberItem("Sensor2"), ZonedDateTime.now(), new DecimalType(1.0), null);
var metaKey = new org.openhab.core.items.MetadataKey("timescaledb", "Sensor2");
var oldMeta = new org.openhab.core.items.Metadata(metaKey, "old", Map.of());
var newMeta = new org.openhab.core.items.Metadata(metaKey, "new", Map.of("aggregation", "MIN"));
service.updated(oldMeta, newMeta);
stubItemIdLookup(6);
reset(dataSource);
when(dataSource.getConnection()).thenReturn(connection);
service.store(new NumberItem("Sensor2"), ZonedDateTime.now(), new DecimalType(3.0), null);
verify(connection, atLeastOnce()).prepareStatement(contains("INSERT INTO item_meta"));
}
@Test
void metadataRemovedInvalidatesCache() throws Exception {
stubItemIdLookup(7);
service.store(new NumberItem("Sensor3"), ZonedDateTime.now(), new DecimalType(1.0), null);
var metaKey = new org.openhab.core.items.MetadataKey("timescaledb", "Sensor3");
var meta = new org.openhab.core.items.Metadata(metaKey, "label", Map.of());
service.removed(meta);
stubItemIdLookup(7);
reset(dataSource);
when(dataSource.getConnection()).thenReturn(connection);
service.store(new NumberItem("Sensor3"), ZonedDateTime.now(), new DecimalType(4.0), null);
verify(connection, atLeastOnce()).prepareStatement(contains("INSERT INTO item_meta"));
}
@Test
void metadataChangeForOtherNamespaceDoesNotInvalidateCache() throws Exception {
stubItemIdLookup(8);
service.store(new NumberItem("Sensor4"), ZonedDateTime.now(), new DecimalType(1.0), null);
// Metadata change in a different namespace — cache must NOT be invalidated
var metaKey = new org.openhab.core.items.MetadataKey("someOtherNamespace", "Sensor4");
var meta = new org.openhab.core.items.Metadata(metaKey, "irrelevant", Map.of());
service.added(meta);
// Reset both mocks so only the second store's interactions are counted
reset(dataSource, connection);
when(dataSource.getConnection()).thenReturn(connection);
PreparedStatement insertItemPs = mock(PreparedStatement.class);
when(connection.prepareStatement(contains("INSERT INTO items"))).thenReturn(insertItemPs);
when(insertItemPs.executeUpdate()).thenReturn(1);
service.store(new NumberItem("Sensor4"), ZonedDateTime.now(), new DecimalType(2.0), null);
// Cache still valid → upsert must NOT have been called in the second store
verify(connection, never()).prepareStatement(contains("INSERT INTO item_meta"));
}
// ------------------------------------------------------------------
// Helpers
// ------------------------------------------------------------------
/**
* Stubs the item_id lookup: SELECT returns nothing, INSERT returns the given id.
* Stubs the item_id UPSERT: INSERT ... ON CONFLICT DO UPDATE ... RETURNING id returns the given id.
*/
private void stubItemIdLookup(int itemId) throws Exception {
ResultSet selectRs = mock(ResultSet.class);
ResultSet insertRs = mock(ResultSet.class);
PreparedStatement selectPs = mock(PreparedStatement.class);
PreparedStatement insertPs = mock(PreparedStatement.class);
ResultSet upsertRs = mock(ResultSet.class);
PreparedStatement upsertPs = mock(PreparedStatement.class);
PreparedStatement insertItemPs = mock(PreparedStatement.class);
when(selectRs.next()).thenReturn(false);
when(insertRs.next()).thenReturn(true);
when(insertRs.getInt(1)).thenReturn(itemId);
when(selectPs.executeQuery()).thenReturn(selectRs);
when(insertPs.executeQuery()).thenReturn(insertRs);
when(upsertRs.next()).thenReturn(true);
when(upsertRs.getInt(1)).thenReturn(itemId);
when(upsertPs.executeQuery()).thenReturn(upsertRs);
when(insertItemPs.executeUpdate()).thenReturn(1);
when(connection.prepareStatement(contains("SELECT id FROM item_meta"))).thenReturn(selectPs);
when(connection.prepareStatement(contains("INSERT INTO item_meta"))).thenReturn(insertPs);
when(connection.prepareStatement(contains("INSERT INTO item_meta"))).thenReturn(upsertPs);
when(connection.prepareStatement(contains("INSERT INTO items"))).thenReturn(insertItemPs);
}
}
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBQueryTest.java
+32 -19
View File
@@ -108,41 +108,54 @@ class TimescaleDBQueryTest {
}
// ------------------------------------------------------------------
// getOrCreateItemId — existing item (SELECT path)
// getOrCreateItemId — UPSERT behaviour
// ------------------------------------------------------------------
@Test
void getOrCreateItemIdExistingitemReturnsfromselect() throws Exception {
void getOrCreateItemIdUsesUpsertAndReturnsId() throws Exception {
when(resultSet.next()).thenReturn(true);
when(resultSet.getInt(1)).thenReturn(42);
int id = TimescaleDBQuery.getOrCreateItemId(connection, "MySensor", "My Sensor Label");
assertEquals(42, id);
// Only one PreparedStatement needed (the SELECT)
verify(connection, times(1)).prepareStatement(contains("SELECT id FROM item_meta"));
verify(connection, times(1)).prepareStatement(contains("INSERT INTO item_meta"));
verify(connection, times(1)).prepareStatement(contains("ON CONFLICT"));
verify(connection, times(1)).prepareStatement(contains("RETURNING id"));
}
@Test
void getOrCreateItemIdNewitemInsertsandreturns() throws Exception {
// First call (SELECT) returns no rows; second (INSERT) returns the new id
ResultSet selectRs = mock(ResultSet.class);
ResultSet insertRs = mock(ResultSet.class);
PreparedStatement selectPs = mock(PreparedStatement.class);
PreparedStatement insertPs = mock(PreparedStatement.class);
void getOrCreateItemIdSetsValueParameter() throws Exception {
when(resultSet.next()).thenReturn(true);
when(resultSet.getInt(1)).thenReturn(7);
when(selectRs.next()).thenReturn(false);
when(insertRs.next()).thenReturn(true);
when(insertRs.getInt(1)).thenReturn(99);
when(selectPs.executeQuery()).thenReturn(selectRs);
when(insertPs.executeQuery()).thenReturn(insertRs);
TimescaleDBQuery.getOrCreateItemId(connection, "Sensor", "label", "sensor.temperature", null);
when(connection.prepareStatement(contains("SELECT id FROM item_meta"))).thenReturn(selectPs);
when(connection.prepareStatement(contains("INSERT INTO item_meta"))).thenReturn(insertPs);
// Parameter 3 = value (TEXT)
verify(preparedStatement).setString(3, "sensor.temperature");
}
int id = TimescaleDBQuery.getOrCreateItemId(connection, "NewSensor", null);
@Test
void getOrCreateItemIdSetsMetadataJsonParameter() throws Exception {
when(resultSet.next()).thenReturn(true);
when(resultSet.getInt(1)).thenReturn(7);
assertEquals(99, id);
String json = "{\"aggregation\":\"AVG\"}";
TimescaleDBQuery.getOrCreateItemId(connection, "Sensor", "label", "sensor.temperature", json);
// Parameter 4 = metadata JSONB — must use setObject with Types.OTHER
verify(preparedStatement).setObject(eq(4), eq(json), eq(java.sql.Types.OTHER));
}
@Test
void getOrCreateItemIdNullValueAndMetadataPassedAsNull() throws Exception {
when(resultSet.next()).thenReturn(true);
when(resultSet.getInt(1)).thenReturn(5);
TimescaleDBQuery.getOrCreateItemId(connection, "Sensor", null);
verify(preparedStatement).setString(3, null);
verify(preparedStatement).setObject(eq(4), isNull(), eq(java.sql.Types.OTHER));
}
// ------------------------------------------------------------------
bundles/org.openhab.persistence.timescaledb/src/test/java/org/openhab/persistence/timescaledb/internal/TimescaleDBSchemaTest.java
+136 -2
View File
@@ -38,19 +38,29 @@ class TimescaleDBSchemaTest {
private Connection connection;
private Statement statement;
private PreparedStatement hypertablePs;
private PreparedStatement verifyPs;
private ResultSet extensionResultSet;
private ResultSet verifyResultSet;
@BeforeEach
void setUp() throws SQLException {
connection = mock(Connection.class);
statement = mock(Statement.class);
hypertablePs = mock(PreparedStatement.class);
verifyPs = mock(PreparedStatement.class);
extensionResultSet = mock(ResultSet.class);
verifyResultSet = mock(ResultSet.class);
when(connection.createStatement()).thenReturn(statement);
when(connection.prepareStatement(anyString())).thenReturn(hypertablePs);
// hypertable PS for create_hypertable; verify PS for schema column check
when(connection.prepareStatement(contains("create_hypertable"))).thenReturn(hypertablePs);
when(connection.prepareStatement(contains("information_schema.columns"))).thenReturn(verifyPs);
when(statement.executeQuery(contains("pg_extension"))).thenReturn(extensionResultSet);
when(extensionResultSet.next()).thenReturn(true); // extension is present by default
// Schema verification: both 'value' and 'metadata' columns present by default
when(verifyPs.executeQuery()).thenReturn(verifyResultSet);
when(verifyResultSet.next()).thenReturn(true, true, false);
when(verifyResultSet.getString(1)).thenReturn("value", "metadata");
}
@Test
@@ -60,9 +70,13 @@ class TimescaleDBSchemaTest {
// Must check TimescaleDB extension
verify(statement).executeQuery(contains("pg_extension"));
// Must create item_meta
// Must create item_meta with metadata column
verify(statement).execute(contains("CREATE TABLE IF NOT EXISTS item_meta"));
// Must run single migration adding both value and metadata columns
verify(statement).execute(argThat(s -> s.contains("ADD COLUMN IF NOT EXISTS value")
&& s.contains("ADD COLUMN IF NOT EXISTS metadata JSONB")));
// Must create items table
verify(statement).execute(contains("CREATE TABLE IF NOT EXISTS items"));
@@ -219,4 +233,124 @@ class TimescaleDBSchemaTest {
assertTrue(migrationSql.indexOf("DROP") < migrationSql.indexOf("ADD"),
"Migration must DROP the old constraint before ADD-ing the new one");
}
// ------------------------------------------------------------------
// item_meta schema — DDL and migration
// ------------------------------------------------------------------
@Test
void createTableItemmetaContainsValueTextColumn() throws SQLException {
var capturedSql = new java.util.ArrayList<String>();
doAnswer(inv -> {
capturedSql.add(inv.getArgument(0));
return false;
}).when(statement).execute(anyString());
TimescaleDBSchema.initialize(connection, "7 days", 0, 0);
java.util.Optional<String> createItemMeta = capturedSql.stream()
.filter(s -> s.contains("CREATE TABLE IF NOT EXISTS item_meta")).findFirst();
assertTrue(createItemMeta.isPresent(), "CREATE TABLE item_meta statement not found");
String ddl = createItemMeta.get();
assertTrue(ddl.contains("value") && ddl.contains("TEXT"),
"CREATE TABLE item_meta must define a 'value TEXT' column");
}
@Test
void createTableItemmetaContainsMetadataJsonbColumn() throws SQLException {
var capturedSql = new java.util.ArrayList<String>();
doAnswer(inv -> {
capturedSql.add(inv.getArgument(0));
return false;
}).when(statement).execute(anyString());
TimescaleDBSchema.initialize(connection, "7 days", 0, 0);
java.util.Optional<String> createItemMeta = capturedSql.stream()
.filter(s -> s.contains("CREATE TABLE IF NOT EXISTS item_meta")).findFirst();
assertTrue(createItemMeta.isPresent(), "CREATE TABLE item_meta statement not found");
String ddl = createItemMeta.get();
assertTrue(ddl.contains("metadata") && ddl.contains("JSONB"),
"CREATE TABLE item_meta must define a 'metadata JSONB' column");
}
@Test
void migrationDdlAddsValueAndMetadataColumnsInSingleStatement() throws SQLException {
var capturedSql = new java.util.ArrayList<String>();
doAnswer(inv -> {
capturedSql.add(inv.getArgument(0));
return false;
}).when(statement).execute(anyString());
TimescaleDBSchema.initialize(connection, "7 days", 0, 0);
// Both columns must be added in a single ALTER TABLE statement (one lock acquisition)
java.util.Optional<String> migrationOpt = capturedSql.stream()
.filter(s -> s.contains("ADD COLUMN IF NOT EXISTS value")
&& s.contains("ADD COLUMN IF NOT EXISTS metadata JSONB"))
.findFirst();
assertTrue(migrationOpt.isPresent(),
"value and metadata columns must be added in a single ALTER TABLE statement");
}
@Test
void migrationDdlUsesLockTimeoutToPreventBlockingServiceStart() throws SQLException {
var capturedSql = new java.util.ArrayList<String>();
doAnswer(inv -> {
capturedSql.add(inv.getArgument(0));
return false;
}).when(statement).execute(anyString());
TimescaleDBSchema.initialize(connection, "7 days", 0, 0);
long migrationStatementsWithoutLockTimeout = capturedSql.stream()
.filter(s -> s.contains("ALTER TABLE item_meta") && !s.contains("lock_timeout")).count();
assertEquals(0, migrationStatementsWithoutLockTimeout,
"All item_meta DDL migrations must use lock_timeout to prevent blocking service startup indefinitely");
}
@Test
void migrationDdlHasExceptionHandlerForLockTimeout() throws SQLException {
var capturedSql = new java.util.ArrayList<String>();
doAnswer(inv -> {
capturedSql.add(inv.getArgument(0));
return false;
}).when(statement).execute(anyString());
TimescaleDBSchema.initialize(connection, "7 days", 0, 0);
long migrationsWithoutExceptionHandler = capturedSql.stream()
.filter(s -> s.contains("item_meta") && s.contains("ALTER TABLE") && !s.contains("EXCEPTION")).count();
assertEquals(0, migrationsWithoutExceptionHandler,
"All item_meta migrations must handle lock_not_available gracefully instead of blocking");
}
@Test
void migrationRunsAfterCreateTableItemMeta() throws SQLException {
var capturedSql = new java.util.ArrayList<String>();
doAnswer(inv -> {
capturedSql.add(inv.getArgument(0));
return false;
}).when(statement).execute(anyString());
TimescaleDBSchema.initialize(connection, "7 days", 0, 0);
int createTableIdx = -1;
int migrationIdx = -1;
for (int i = 0; i < capturedSql.size(); i++) {
String s = capturedSql.get(i);
if (s.contains("CREATE TABLE IF NOT EXISTS item_meta") && createTableIdx < 0) {
createTableIdx = i;
}
if (s.contains("ADD COLUMN IF NOT EXISTS value") && s.contains("ADD COLUMN IF NOT EXISTS metadata JSONB")
&& migrationIdx < 0) {
migrationIdx = i;
}
}
assertTrue(createTableIdx >= 0, "CREATE TABLE item_meta must be executed");
assertTrue(migrationIdx >= 0, "Column migration must be executed");
assertTrue(createTableIdx < migrationIdx,
"Migration must run after CREATE TABLE item_meta so that the ALTER runs on an existing table");
}
}