Okay, that is not directly related OH integration then. It is in JSON parsing library, but OH integration is using an older parser. Probably, some other integrations (if you have any), or other parts of the app cause that.
I donât have any other integrations. It is being called with âcom.habitautomated.shdp.value.AutoValue_Deviceâ as Argument quite often, also multiple times per second (about 100 times per second). Is that new parser activated with the SHDP OH Integration option from the Labs section? I havenât seen any mention what âSHDP OH Integrationâ is or does, itâs disabled in my case.
No, OH integration does not support new parser yet.
Btw, if you see that in Labs section, it means you have developer mode enabled, which will slow down the app in some places because of extra logging. It should not be enable-able through the normal UI.
Just tried a brand-new OH installation and a brand new HomeHabit installation (so developer mode off), added 100 DateTime items with NTP to OH, 1 Dimmer and 1 Switch, Homehabit got 1 Value item to show the time (itâs updated properly), one slider for the Dimmer and one Switch for the Dimmer. If I turn on the Dimmer with the Switch itâs stuck in âTurning onâŚâ and only after some time it is updated (along with the slider). I donât see time lagging behind or skipping seconds though, just the Dimmer update getting missed or delayed. Just to confirm: You are not seeing the same behaviour? You are seeing instant updates, even when you use the Release Version I am using? Maybe you need to wait a little (like a minute or two), it seems like itâs fast immediately after a (re)connect and then getâs slower over time until it stops working completely.
When I put the App in the background and then in the foreground again the state is updated, I think the connection is re-established in that case?
Managed to reproduce this behavior with artificially multiplying events coming from OH.
With 120 DateTime items updating every second and repeated 10 times each, so 1200 events per second, it started skipping some seconds. With 120 events x 50 = 6000 per second, it was skipping most of the time, and switches were loosing state updates.
Since you were saying that most of events coming from OH in your case are updates and not changes, I think there is a solution that would help. I believe event processing can handle higher volume, but each item update triggers other things, including database save and that is probably the issue here.
I will add a filter that would exclude events without changes to the state at the first possible opportunity. Letâs see if that will help.
Youâre artificially building the statechanged option in OH now 
Iâd probably be happy if that is used, comparing states seems to be additional overhead again, not sure if any other integration emits âuselessâ events.
OH integration code already keeps cache of all current items, since events only have partial data, so this change is minimal, and hopefully helps in the short term.
Removing ItemStateEvent is still desired, but it would require more testing, so will be planned for a later release.
Awesome, sounds good.
Are you already preventing duplicate Updates? If 10 updates are âqueuedâ only the last one needs to be processed, so all the other previous ones can be skipped in that case. I know that I have some pumps that report their RPMs multiple times per second, but what we really care about is the last one. Iâm not sure what the best way to implement this would be so an event can be updated without losing its position in the queue (then it would âstarveâ and never be updated).
I just found the json library youâre most likely using, have you tried the @CompiledJson annotation for the class I mentioned? Itâs supposed to avoid reflection, and I think that would also avoid that expensive call (and probably accelerate conversion aswell).
The latest fix is available in 28.9 release.
No, all item updates are treated the same.
Yes, that annotation is used.
I am a little surprised that there is not DslJsonConverter created for the AutoValueDevice, there are converters for Device, DeviceUpdate, Binding and so on, but not for the AutoValue* things that are extending some other classes (at least in 28.8, 28.9 isnât here yet). That seems to be another bottleneck, and it seems to be for item saving in the database? Then optimizing this so the converter is created (I donât really understand why it isnât if there is an annotation in com.habitautomated.shdp.value.AutoValue_Device) would probably help increasing performance aswell.
Iâve got nothing worthwhile to add except Iâm loving the collaboration depth of investigation here! Nice work!
1 Like
I just had a chance to try and look at the latest version. Again it is a huge improvement, CPU at around 40% now (so almost as good as my solution) and more importantly: The issue seems to be finally gone!
One thing I noticed though is a very high i/o activity (also on the older versions): Homehabit is writing almost constantly 6MB/s to the flash. That is not good at all, as it causes excessive wear for the flash. Is it really necessary to store the state of the items persistent across App-restarts? They will most likely be outdated anyways when the App is restarted, so I donât see the point in doing that? I suggest either keeping the current state in-memory only or keep it in memory and only commit the states once every 5 minutes (or only when the App is exited). That would further reduce CPU usage and will ensure that the flash is not killed due to excessive writes (6MB/s is a lot! Even if some of that is caught by the Operating Systemâs caching this is not good at all).
As a test I moved the entire database-directory to a RAM-Disk and that reduced the memory I/O to basically 0. Of course I will lose any changes across reboots of my device but thatâs not an issue for me at the moment. If the eMMC dies from this I would have much bigger issues.
Thatâs great!
There are several reasons why item states are persisted. Just of the top of my head: some integrations cannot be refreshed every time the app is launched, and keeping all item states in memory can be a performance issue on slower devices.
I will try to reproduce the same writing to storage numbers that you mentioned, and see if there are any unnecessary writes.
Wouldnât this prevent other changes from saving as well, like widgets or any other configuration?
There might be possible issues because of mismatch between old database and an upgraded app version.
Yes I can no longer change anything unless I write back the database to the storage, or at least itâs not persistent across device reboots. I could probably do that copying back automatically once per day, but right now it also acts as a protection against misconfiguration/accidental changes
Iâm not really sure what the best way to solve it would be, while my idea is working itâs still a rather strange solution. Maybe only persist states for items that canât be refreshed (those should also be the ones that are rarely changing, so it shouldnât really matter)? Probably some Integrations could simply be exempt from saving the state, like openHAB. I donât think it would take much memory to keep the states in RAM, my database is just 3MB (and I have over 1100 items), a hash map would probably use less memory.