[blockly] Persistence: Enhance existing blocks & add blocks to cover all available actions

[mherwege]

Closes #2595.

This PR further extends the Blockly capabilities to include more of the persistence extensions.

Specifically enhanced the oh_get_persistvalue block:

• Add all Between methods of already existing methods
• Add countSince, countUntil and countBetween
• Add countStateChangesSince, countStateChangesUntil and countStateChangesBetween
• Add getAllStatesSince, getAllStatesUntil, getAllStatesBetween returning an Array of timestamp and state pairs
• Add a parameter for return type if relevant (String, Number, Quantity, Timestamp). These will be shown if relevant for the specific method (and limited to the relevant ones). The sorting of the options in the dropdown is such that the first in the list will generate code that is backward compatible. But it now allows to retrieve all of the return information, including for QuantityTypes. (only available for GraalJS)

Added methods to the oh_get_persistence_lastupdate block (depends on: openhab/openhab-js#350)

• lastChange, nextChange

Created a new block oh_delete_persistedvalues to delete persisted values:

• deleteAllStatesSince, deleteAllStatesUntil, deleteAllStatesBetween

Created a new block oh_persist to persist values:

• currentState: persists a state at the current time
• stateAt: persists a state to a given time
• stateList: persists a TimeSeries

Initial testing shows this is generating the correct code.

EDIT: Resolved the issue below.
I have however identified one issue I was not able to resolve: depending on the persistence method, I need 0, 1 or 2 date fields. By default, there are only shadow blocks shown for the date field available when dropping the block on the canvas initially. As the first method in the list has only 1 date parameter, even when switching the method to one requiring 2, there is still only one shadow block shown. Switching to a method with 0 date field parameters and then back to a method with 1 or 2 even completely removes the shadow blocks. I don't know if and how this can be solved.

EDIT: As this is not being reviewed yet, I completed the development to also include the persistence actions that were not covered yet. I therefore created 2 extra blocks and extended the existing block.

[relativeci]

#2083 Bundle Size — 10.68MiB (+0.15%).

e74adf2(current) vs 32688a5 main#2082(baseline)

Warning

Bundle contains 2 duplicate packages – View duplicate packages

Bundle metrics   1 change

	Current #2083	Baseline #2082
Initial JS	1.88MiB	1.88MiB
Initial CSS	607.91KiB	607.91KiB
Cache Invalidation	18.2%	24.45%
Chunks	223	223
Assets	246	246
Modules	2894	2894
Duplicate Modules	149	149
Duplicate Code	1.85%	1.85%
Packages	97	97
Duplicate Packages	2	2

Bundle size by type   1 change 1 regression

	Current #2083	Baseline #2082
JS	8.86MiB (+0.19%)	8.85MiB
CSS	891.55KiB	891.55KiB
Fonts	526.1KiB	526.1KiB
Media	295.6KiB	295.6KiB
IMG	140.74KiB	140.74KiB
HTML	1.24KiB	1.24KiB
Other	871B	871B

---

Bundle analysis report Branch mherwege:blockly_persistence Project dashboard

[mherwege]

@stefan-hoehn Could you please help me with the shadow block issue? Apart from that, I think this is complete.
EDIT: This issue has been resolved.

@florian-h05 Would the approach I took here keep the backward compatibility as you had in mind?

[mherwege]

@stefan-hoehn Could you please help me with the shadow block issue? Apart from that, I think this is complete.

OK, I figured it out. It should work now.

[stefan-hoehn]

@mherwege First of all, I amazed that you provided this rather comprehensive change in blockly, so Kudos. Just to set expectations, I wasn't able to look into github the last week and I won't be able to review and test this in the next 3 weeks due to absence.
In the case of persistence I have a range of test rules that I usually have to run and check and also add the new methods to it, so it is not only about reviewing the code which I don't have the time today.
So either @florian-h05 does it (but please make sure you test all the variations) or it has to wait until I am back 😢

[mherwege]

@stefan-hoehn Thank you for the Kudos. No worries if this gets delayed. I guess it may have to wait for afte 4.2 in that case if @florian-h05 is not reviewing. But I am OK with that.

[mherwege]

Some example code for the most recent additions:

Generates:

var tempList, persistedState, newTempList;


tempList = items.getItem('Local_Weather_and_Forecast_One_Call_API_Outdoor_Temperature').persistence.getAllStatesUntil(time.ZonedDateTime.now().plusWeeks(1), 'inmemory').map(v => ([v.timestamp, v.state]));
console.info('Original retrieved temperature list');
for (var persistedState_index in tempList) {
  persistedState = tempList[persistedState_index];
  console.info((['  state: ',persistedState[0],' -> ',persistedState[1]].join('')));
}
items.getItem('TemperatureCopy').persistence.persist(time.ZonedDateTime.now().minusHours(1), '0 °C', 'inmemory');
var timeSeries = new items.TimeSeries('REPLACE');
tempList.forEach(s => timeSeries.add(s[0], s[1]));
items.getItem('TemperatureCopy').persistence.persist(timeSeries, 'inmemory');
newTempList = items.getItem('TemperatureCopy').persistence.getAllStatesBetween(time.ZonedDateTime.now().minusWeeks(1) ,time.ZonedDateTime.now().plusWeeks(1), 'inmemory').map(v => ([v.timestamp, v.state]));
console.info('Stored and retrieved temperature list');
for (var persistedState_index2 in newTempList) {
  persistedState = newTempList[persistedState_index2];
  console.info((['  state: ',persistedState[0],' -> ',persistedState[1]].join('')));
}
items.getItem('TemperatureCopy').persistence.removeAllStatesSince(time.ZonedDateTime.now().minusHours(3), 'inmemory');
newTempList = items.getItem('TemperatureCopy').persistence.getAllStatesBetween(time.ZonedDateTime.now().minusWeeks(1) ,time.ZonedDateTime.now().plusWeeks(1), 'inmemory').map(v => ([v.timestamp, v.state]));
console.info('Retrieved after delete');
for (var persistedState_index3 in newTempList) {
  persistedState = newTempList[persistedState_index3];
  console.info((['  state: ',persistedState[0],' -> ',persistedState[1]].join('')));
}
items.getItem('TemperatureCopy').persistence.removeAllStatesSince(time.ZonedDateTime.now().minusWeeks(3), 'inmemory');

[ghys]

I have one remark:

items.getItem('TemperatureCopy').persistence.persist(time.ZonedDateTime.now().minusHours(1), '0 °C', 'inmemory');

I'm confused with this block, when it says "persist state at a specific time" I would expect a time and not a temperature.

[mherwege]

I'm confused with this block, when it says "persist state at a specific time" I would expect a time and not a temperature.

I understand the confusion. The oh_persist block as I designed it always has the state as the first field, also when persisting at current time or a series. Unless I make a separate field dor the time selection (which means I will have an extra line in the block when I persist at current time, making the block larger), that one dropdown will have to represent the meaning. Do you have a suggestion for a better label?

[stefan-hoehn]

I think it looks good and I am happy you provided it. I feel that persist value of item at time IMHO is okay and actually have no better idea atm. Therefore 🤘👍 from my side

[mherwege]

Thinking about it, maybe ‘state (at current time)’ and ‘state (at specified time)’ would be clearer.

[florian-h05]

LGTM, thanks!

Wrt to the remark about the block:
I agree that it is not optimal, but due to lack of better ideas I will merge this now - the added value outweighs this minor remark.

[florian-h05]

Thinking about it, maybe ‘state (at current time)’ and ‘state (at specified time)’ would be clearer.

Can you commit right now? If not, I can do it.

[florian-h05]

@mherwege I will push this minor change now so we can merge this and start the build soon.

[mherwege]

@mherwege I will push this minor change now so we can merge this and start the build soon.

Thanks.