Hi Iain.
Thanks you for tanking us on your journey.
I have read your posts with interest and I'm sure others have as well
So Obvious.......
-
medelec35
- Valued Contributor
- Posts: 2352
- http://meble-kuchenne.info.pl
- Joined: Wed Dec 02, 2020 11:07 pm
- Has thanked: 733 times
- Been thanked: 793 times
-
chipfryer27
- Valued Contributor
- Posts: 2069
- Joined: Thu Dec 03, 2020 10:57 am
- Has thanked: 453 times
- Been thanked: 677 times
Re: So Obvious.......
Thanks Martin
Once working it is my intent to give a block diagram of the circuit and post my chart.
Not sure yet if I'm going to do something with App Developer as MQTT Alert does what I need, although not exactly as I'd like.
Regards
Once working it is my intent to give a block diagram of the circuit and post my chart.
Not sure yet if I'm going to do something with App Developer as MQTT Alert does what I need, although not exactly as I'd like.
Regards
-
chipfryer27
- Valued Contributor
- Posts: 2069
- Joined: Thu Dec 03, 2020 10:57 am
- Has thanked: 453 times
- Been thanked: 677 times
Re: So Obvious.......
Hi
As the primary objective in all of this is to document activity within my house I need a way to store it and my RAMP is ideal. Not having used it it a while I needed to refamiliarise myself. I posted a guide ages ago and following it things quickly came back.
Biggest problems was what to name my database, what table(s) should it contain and what data should be in the tables? No doubt I'll change what I have but it will do for now. My Fields include ID number, Timestamp, Device, Location and Status which is updated by Flowcode.
Simulating it all worked relatively OK, updating both MQTT and database and any issues I saw I put down to simulation.
Before mounting everything to the wall, I thought to test the code on hardware...... Not good.......
I remember now why that ESP was sitting doing nothing..... It's truly useless.... It has a wifi range of about 50 centimetres.... I don't know why I didn't through it away when I discovered it, but it has came back to bite me now.
Too late in the day to mess further, especially as a nice red is desperately wanting to come out and play...
Regards
As the primary objective in all of this is to document activity within my house I need a way to store it and my RAMP is ideal. Not having used it it a while I needed to refamiliarise myself. I posted a guide ages ago and following it things quickly came back.
Biggest problems was what to name my database, what table(s) should it contain and what data should be in the tables? No doubt I'll change what I have but it will do for now. My Fields include ID number, Timestamp, Device, Location and Status which is updated by Flowcode.
Simulating it all worked relatively OK, updating both MQTT and database and any issues I saw I put down to simulation.
Before mounting everything to the wall, I thought to test the code on hardware...... Not good.......
I remember now why that ESP was sitting doing nothing..... It's truly useless.... It has a wifi range of about 50 centimetres.... I don't know why I didn't through it away when I discovered it, but it has came back to bite me now.
Too late in the day to mess further, especially as a nice red is desperately wanting to come out and play...
Regards
-
chipfryer27
- Valued Contributor
- Posts: 2069
- Joined: Thu Dec 03, 2020 10:57 am
- Has thanked: 453 times
- Been thanked: 677 times
Re: So Obvious.......
Hi
Reviews of Elegoo seem they're reasonable as a supplier and as Jaycar stock some their products (a sign of some quality) I took the plunge and ordered up a replacement ESP. Two for around £7 or so from usual sites. Build quality from a visual inspection is high so high hopes of this being more than an annoyance.
Downloading code it connected straight away through walls. The previous wouldn't do more than a few tens of centimetres, so a big improvement there. However I ran into WDT issues. I know the ESP's are tricky regarding such and that you need to add in delays at strategic places (especially within loops) to keep them happy. I'm not a fan of espressif so certainly no expert with their products but this was unexpected. Usually a delay of 20mS or so strategically placed does the job, but I was having big problems.
Cutting a long, investigative story short, I ended up recreating my chart and adding in components one at a time to establish where the issue lay. As I added in switch / switch array components a clue sprang forth. Their default debounce is 5mS which is quite a short time for mechanical connections. I had upped this to 20mS, which is really borderline acceptable.
My issues disappeared when I reduced this on my original chart from 20mS to 5mS and it then became vary stable. During subsequent testing I was not troubled by WDT at all.
Chatting with Martin (all hail), he informed that these components use multiples of 1mS delays to give your debounce value, so a debounce of 20mS isn't a "delay" of 20mS but 20 x 1mS.
So be aware of this as another potential "gotcha" when using ESP.
Note that my chart was doing other things too which was eating up time before the necessary cede, so debounce may not be an issue for you.
My testing does show my chart is doing what I need, but as always now I have it running, I see a better way of doing things. Focus now is on MQTT and getting it to accept my "key presses" from afar. Basic testing works, just need to refine. Chart does capture and provide the correct value but I'm rethinking things again.
Regards
Reviews of Elegoo seem they're reasonable as a supplier and as Jaycar stock some their products (a sign of some quality) I took the plunge and ordered up a replacement ESP. Two for around £7 or so from usual sites. Build quality from a visual inspection is high so high hopes of this being more than an annoyance.
Downloading code it connected straight away through walls. The previous wouldn't do more than a few tens of centimetres, so a big improvement there. However I ran into WDT issues. I know the ESP's are tricky regarding such and that you need to add in delays at strategic places (especially within loops) to keep them happy. I'm not a fan of espressif so certainly no expert with their products but this was unexpected. Usually a delay of 20mS or so strategically placed does the job, but I was having big problems.
Cutting a long, investigative story short, I ended up recreating my chart and adding in components one at a time to establish where the issue lay. As I added in switch / switch array components a clue sprang forth. Their default debounce is 5mS which is quite a short time for mechanical connections. I had upped this to 20mS, which is really borderline acceptable.
My issues disappeared when I reduced this on my original chart from 20mS to 5mS and it then became vary stable. During subsequent testing I was not troubled by WDT at all.
Chatting with Martin (all hail), he informed that these components use multiples of 1mS delays to give your debounce value, so a debounce of 20mS isn't a "delay" of 20mS but 20 x 1mS.
So be aware of this as another potential "gotcha" when using ESP.
Note that my chart was doing other things too which was eating up time before the necessary cede, so debounce may not be an issue for you.
My testing does show my chart is doing what I need, but as always now I have it running, I see a better way of doing things. Focus now is on MQTT and getting it to accept my "key presses" from afar. Basic testing works, just need to refine. Chart does capture and provide the correct value but I'm rethinking things again.
Regards
-
chipfryer27
- Valued Contributor
- Posts: 2069
- Joined: Thu Dec 03, 2020 10:57 am
- Has thanked: 453 times
- Been thanked: 677 times
Re: So Obvious.......
Hi
Connected a scope to a zone input on my control panel and 0v.
Panel input is normally high, going low when a PIR / detector triggers (N/C detector) and I set my scope for a single shot on a falling edge.
As you can see a clean transition so no real need for any debounce in my chart.
Regards
Connected a scope to a zone input on my control panel and 0v.
Panel input is normally high, going low when a PIR / detector triggers (N/C detector) and I set my scope for a single shot on a falling edge.
As you can see a clean transition so no real need for any debounce in my chart.
Regards
-
chipfryer27
- Valued Contributor
- Posts: 2069
- Joined: Thu Dec 03, 2020 10:57 am
- Has thanked: 453 times
- Been thanked: 677 times
Re: So Obvious.......
Hi
A quick update.
I replaced the dodgy ESP with one from a more reputable source and it seems to be working fine. Major pain to remove the old ESP as the Vero tracks were starting to lift, so out came the Dremel with a cutting disc. Slicing the board along the edge of the pins it was soon in the bin. Further cutting of the pins into smaller sections allowed their removal with relative ease.
I then modified a 40-pin socket by cutting it in half then cutting for the correct number of pins. Once soldered to the board I can now insert / remove the ESP with ease.
Forgetting my 12-wire alarm cable was still in my Evil-Lab, I could only connect in the "keypad" connections and power from USB. No issue for my testing.
Very pleased to say that that it worked first time and I can send "key presses" from my laptop to the ESP via my MQTT broker and subsequently control my alarm. Whatever I can do on the actual keypad I can do via MQTT. Incidently after every message is processed the ESP sends (via MQTT) a status update informing of Set/Unset and Triggered / Normal.
I have found a few issues with the ESP and Flowcode MQTT components which I'm hesitant to call a bug as it may just be me doing yet again something stupid.
I bought a twin pack of ESPs so lets call them ESP1 and ESP2 as my creativity is low today..
The FC component "Connect to SSID" should return 1 when / if connected.
Previously to check if I were connected I could just tell it to connect again and off it would go, returning a 1.
On ESP1 this is the case but on ESP2 if it is already connected it pumps out a warning over UART, waits for the timeout to expire then returns 0. So not a good way to establish if you are already connected. I think this is again more to do with espressif changing things than FC being at fault.
I tried the MQTT Ping command which returns a non-zero value if successful. This does seem to work up until you have a message to read. It then returns 0 if there is one waiting to be read, so again not a good way to establish if you are connected.
In my "Read" loop, if there is a message to be read it obtains and processes. However if I don't include a command to resubscribe after the message is processed then the chart ignores all future messages.
I'm going to look into the above further, with just basic code to test, just in case it is me that has done something stupid in this chart.
It would be nice if we had option in the MQTT component to set clean start to True / False and also the QoS of messages 0/1/2.
Regards
A quick update.
I replaced the dodgy ESP with one from a more reputable source and it seems to be working fine. Major pain to remove the old ESP as the Vero tracks were starting to lift, so out came the Dremel with a cutting disc. Slicing the board along the edge of the pins it was soon in the bin. Further cutting of the pins into smaller sections allowed their removal with relative ease.
I then modified a 40-pin socket by cutting it in half then cutting for the correct number of pins. Once soldered to the board I can now insert / remove the ESP with ease.
Forgetting my 12-wire alarm cable was still in my Evil-Lab, I could only connect in the "keypad" connections and power from USB. No issue for my testing.
Very pleased to say that that it worked first time and I can send "key presses" from my laptop to the ESP via my MQTT broker and subsequently control my alarm. Whatever I can do on the actual keypad I can do via MQTT. Incidently after every message is processed the ESP sends (via MQTT) a status update informing of Set/Unset and Triggered / Normal.
I have found a few issues with the ESP and Flowcode MQTT components which I'm hesitant to call a bug as it may just be me doing yet again something stupid.
I bought a twin pack of ESPs so lets call them ESP1 and ESP2 as my creativity is low today..
The FC component "Connect to SSID" should return 1 when / if connected.
Previously to check if I were connected I could just tell it to connect again and off it would go, returning a 1.
On ESP1 this is the case but on ESP2 if it is already connected it pumps out a warning over UART, waits for the timeout to expire then returns 0. So not a good way to establish if you are already connected. I think this is again more to do with espressif changing things than FC being at fault.
I tried the MQTT Ping command which returns a non-zero value if successful. This does seem to work up until you have a message to read. It then returns 0 if there is one waiting to be read, so again not a good way to establish if you are connected.
In my "Read" loop, if there is a message to be read it obtains and processes. However if I don't include a command to resubscribe after the message is processed then the chart ignores all future messages.
I'm going to look into the above further, with just basic code to test, just in case it is me that has done something stupid in this chart.
It would be nice if we had option in the MQTT component to set clean start to True / False and also the QoS of messages 0/1/2.
Regards
-
Steve-Matrix
- Matrix Staff
- Posts: 2007
- Joined: Sat Dec 05, 2020 10:32 am
- Has thanked: 300 times
- Been thanked: 465 times
Re: So Obvious.......
Thanks for keeping updated on your adventure!
"Clean Start" is a MQTT v5.0 setting.
The Web Developer component uses MQTT v3.1 or v3.1.1 and has the "Clean Session" flag as a property of the component, and also the QoS setting as a parameter when publishing or subscribing to a message.
I'm not as familiar with the PC Developer and Embedded components for MQTT. Hopefully Ben can answer that one for you.
"Clean Start" is a MQTT v5.0 setting.
The Web Developer component uses MQTT v3.1 or v3.1.1 and has the "Clean Session" flag as a property of the component, and also the QoS setting as a parameter when publishing or subscribing to a message.
I'm not as familiar with the PC Developer and Embedded components for MQTT. Hopefully Ben can answer that one for you.
-
chipfryer27
- Valued Contributor
- Posts: 2069
- Joined: Thu Dec 03, 2020 10:57 am
- Has thanked: 453 times
- Been thanked: 677 times
Re: So Obvious.......
Hi Steve
Thanks for that. I'm a bit busy at present so only doing what I can when I can. The way I have it at the moment is that my ESP based "monitor" communicates with my MQTT broker over my LAN, and it is very stable and reliable so I am not worried at all that within my LAN any message may be lost. Highly unlikely and of no consequence if they are as it would not impede the security system in any way.
What would be nice is that my phone won't be on / reachable 24/7 so would not be able to receive all potential messages, therefore queuing would be nice. I hadn't yet got to creating my on Web App so hadn't noticed WD already has the features. However as the ESP still needs to send the QoS it still leaves a gap.
If I am really bothered about it I could simply have the ESP trigger a script on my RAMMP (extra "M" for Mosquito <s>) that then sends the message.
My phone is only used to control the Alarm but mainly get status updates when Set/Triggered, so I could simply send a null string (perhaps a button called "Status" or such like) which the panel will ignore, but as I have the panel returning Status after each comms I would get my update.
Very much work and thoughts in progress <s>
Regards
Thanks for that. I'm a bit busy at present so only doing what I can when I can. The way I have it at the moment is that my ESP based "monitor" communicates with my MQTT broker over my LAN, and it is very stable and reliable so I am not worried at all that within my LAN any message may be lost. Highly unlikely and of no consequence if they are as it would not impede the security system in any way.
What would be nice is that my phone won't be on / reachable 24/7 so would not be able to receive all potential messages, therefore queuing would be nice. I hadn't yet got to creating my on Web App so hadn't noticed WD already has the features. However as the ESP still needs to send the QoS it still leaves a gap.
If I am really bothered about it I could simply have the ESP trigger a script on my RAMMP (extra "M" for Mosquito <s>) that then sends the message.
My phone is only used to control the Alarm but mainly get status updates when Set/Triggered, so I could simply send a null string (perhaps a button called "Status" or such like) which the panel will ignore, but as I have the panel returning Status after each comms I would get my update.
Very much work and thoughts in progress <s>
Regards
-
chipfryer27
- Valued Contributor
- Posts: 2069
- Joined: Thu Dec 03, 2020 10:57 am
- Has thanked: 453 times
- Been thanked: 677 times
Re: So Obvious.......
Hi
Just a brief update.
With everything connected I couldn't program the ESP. I thought I had avoided all problematic pins and couldn't for the life of me see what I had done wrong. Martin (all hail) quickly informed that GPIO12 was used during boot, something that although I had the datasheet open in front of me failed to see.
As this is connected to a currently unused zone I will just disconnect and ignore. I have made so many "improvements" since starting this, the interface looks like a rat's nest made inside a pair of Heath Robinson's discarded underwear..... When I have more time a custom PCB will be milled and used, but the current board has served well as a development platform.
As far as monitoring the security system detectors it works very well. I created two databases on my RAMMP, one for the alarm panel status (set/unset, alarmed/normal) and the other logging detector activity.
The original intent of all of this was to capture activity which could be used elsewhere and it is doing this without any issue. I didn't realise that whilst sleeping, every 90 minutes or so I move around enough to trigger the PIR..... Having a database full of activity will be useful in detecting patterns and deviation.
I could even use is as a "get out of your chair you lazy person" health device by beeping the chimes on the alarm to force me to move if I have been inactive for too long
Other applications could include "are you OK" alerts. If the last activity was say the hall, then nothing else for a predefined time (and panel wasn't then set indicating house vacated) then that would be unusual and action could be taken. I'm sure you get the idea.
The status of the panel, Set / Triggered, is sent to my phone via MQTT and I can control from my phone the panel again using MQTT. I can set/unset and request a status update. I'm using a NoIP address as target IP which then leads to my RAMMP.
There are a couple of bugs, as expected. One of which I need to connect a scope to the panel to identify cause, but I'm expecting I can resolve in software if it is indeed due to what I think.
Software will be rewritten too, as how I initially planned to do things has been modified so much as "improvements" became apparent, that it is better to just rewrite as that will be easier to maintain. Martin has provided some examples of using MQTT with external libraries which I'll explore when I have more time.
I also intend to create my own app using FC WD, but until then I seem to have settled on "IoT MQTT Panel" on my android devices as it provides an easy to use interface. Currently I have icons for Armed and Triggered that change (including colour) depending on alarm system status. I have two buttons that Arm / Disarm and a third button that requests a Status update.
As mentioned I do have a couple of bugs but no showstopper so far.
Regards
Just a brief update.
With everything connected I couldn't program the ESP. I thought I had avoided all problematic pins and couldn't for the life of me see what I had done wrong. Martin (all hail) quickly informed that GPIO12 was used during boot, something that although I had the datasheet open in front of me failed to see.
As this is connected to a currently unused zone I will just disconnect and ignore. I have made so many "improvements" since starting this, the interface looks like a rat's nest made inside a pair of Heath Robinson's discarded underwear..... When I have more time a custom PCB will be milled and used, but the current board has served well as a development platform.
As far as monitoring the security system detectors it works very well. I created two databases on my RAMMP, one for the alarm panel status (set/unset, alarmed/normal) and the other logging detector activity.
The original intent of all of this was to capture activity which could be used elsewhere and it is doing this without any issue. I didn't realise that whilst sleeping, every 90 minutes or so I move around enough to trigger the PIR..... Having a database full of activity will be useful in detecting patterns and deviation.
I could even use is as a "get out of your chair you lazy person" health device by beeping the chimes on the alarm to force me to move if I have been inactive for too long
Other applications could include "are you OK" alerts. If the last activity was say the hall, then nothing else for a predefined time (and panel wasn't then set indicating house vacated) then that would be unusual and action could be taken. I'm sure you get the idea.
The status of the panel, Set / Triggered, is sent to my phone via MQTT and I can control from my phone the panel again using MQTT. I can set/unset and request a status update. I'm using a NoIP address as target IP which then leads to my RAMMP.
There are a couple of bugs, as expected. One of which I need to connect a scope to the panel to identify cause, but I'm expecting I can resolve in software if it is indeed due to what I think.
Software will be rewritten too, as how I initially planned to do things has been modified so much as "improvements" became apparent, that it is better to just rewrite as that will be easier to maintain. Martin has provided some examples of using MQTT with external libraries which I'll explore when I have more time.
I also intend to create my own app using FC WD, but until then I seem to have settled on "IoT MQTT Panel" on my android devices as it provides an easy to use interface. Currently I have icons for Armed and Triggered that change (including colour) depending on alarm system status. I have two buttons that Arm / Disarm and a third button that requests a Status update.
As mentioned I do have a couple of bugs but no showstopper so far.
Regards
-
Steve-Matrix
- Matrix Staff
- Posts: 2007
- Joined: Sat Dec 05, 2020 10:32 am
- Has thanked: 300 times
- Been thanked: 465 times
Re: So Obvious.......
Thanks for the update - it's really interesting.
When you come to use Flowcode Web Developer, let us know if there are things you'd like to do that are not already there. For example, other components or better information on doing certain things with it.
We have plans for improving this mode of Flowcode anyway, but it is always good to have practical projects in mind when steering development.
When you come to use Flowcode Web Developer, let us know if there are things you'd like to do that are not already there. For example, other components or better information on doing certain things with it.
We have plans for improving this mode of Flowcode anyway, but it is always good to have practical projects in mind when steering development.