ESP8266, MQTT, ThingSpeak
Moderator: Benj
- AbhijitR
- Posts: 300
- Joined: Fri Nov 07, 2014 12:48 pm
- Location: Pune, India
- Has thanked: 283 times
- Been thanked: 80 times
ESP8266, MQTT, ThingSpeak
Hello!
Since last couple of days i am working on one of the projects from MQTT Client (Component)
https://www.matrixtsl.com/wiki/images/c ... P8266.fcfx
I made the necessary changes in the chart as i wanted, on switching on the hardware i noticed the wifi module is getting connected to my mobile hotspot (i can see that in the connected devices in my phone), so one thing is sure the chart is working well until getting in the loop or initializing is done correctly, the SSID macro is working correctly as the name and password working good for the hotspot.
But further the Connected decision is not working as i do not see the LED added to the C0 port getting ON in that branch, what probable mistake i am doing can someone help. Did i miss any properties to be punched in for any component, i have also created an account on thingspeak to see the analog value but nothing is displayed on the plot.
No doubt i am doing some mistake but i did not find any help on google (Flowcode+Thingspeak+MQTT) nor forum, or probably i missed the correct words while searching, this is new for me to understand. In one of the videos i learned it is needed to punch the API key but again where in Flowcode i did not understand, can someone give me some hint or spare some time to check my chart, it will be a great help.
Thank you.
Abhi
Since last couple of days i am working on one of the projects from MQTT Client (Component)
https://www.matrixtsl.com/wiki/images/c ... P8266.fcfx
I made the necessary changes in the chart as i wanted, on switching on the hardware i noticed the wifi module is getting connected to my mobile hotspot (i can see that in the connected devices in my phone), so one thing is sure the chart is working well until getting in the loop or initializing is done correctly, the SSID macro is working correctly as the name and password working good for the hotspot.
But further the Connected decision is not working as i do not see the LED added to the C0 port getting ON in that branch, what probable mistake i am doing can someone help. Did i miss any properties to be punched in for any component, i have also created an account on thingspeak to see the analog value but nothing is displayed on the plot.
No doubt i am doing some mistake but i did not find any help on google (Flowcode+Thingspeak+MQTT) nor forum, or probably i missed the correct words while searching, this is new for me to understand. In one of the videos i learned it is needed to punch the API key but again where in Flowcode i did not understand, can someone give me some hint or spare some time to check my chart, it will be a great help.
Thank you.
Abhi
- Attachments
-
- IOT_Project_18F27K42_25-05-2023.fcfx
- (16.05 KiB) Downloaded 279 times
- Benj
- Matrix Staff
- Posts: 15312
- Joined: Mon Oct 16, 2006 10:48 am
- Location: Matrix TS Ltd
- Has thanked: 4803 times
- Been thanked: 4314 times
- Contact:
Re: ESP8266, MQTT, ThingSpeak
Hello Abhi,
Investigating for you now.
Please do not share login details.
Investigating for you now.
Please do not share login details.
Regards Ben Rowland - MatrixTSL
Flowcode Product Page - Flowcode Help Wiki - Flowcode Examples - Flowcode Blog - Flowcode Course - My YouTube Channel
Flowcode Product Page - Flowcode Help Wiki - Flowcode Examples - Flowcode Blog - Flowcode Course - My YouTube Channel
- Benj
- Matrix Staff
- Posts: 15312
- Joined: Mon Oct 16, 2006 10:48 am
- Location: Matrix TS Ltd
- Has thanked: 4803 times
- Been thanked: 4314 times
- Contact:
Re: ESP8266, MQTT, ThingSpeak
In the MQTT Client component you maybe need to turn the Authentication property to On and then enter your thingspeak username and password.
This is also interesting.
This is also interesting.
This means when you publish daat you won't get an acknowledge signal meaning your program will go into disconnected mode and then re-connect. This when working correctly should not be required.ThingSpeak implements MQTT publish and subscribe functionality with QoS = 0. This "fire-and-forget" protocol means that your client does not receive an acknowledgment when you publish to a ThingSpeak channel.
Regards Ben Rowland - MatrixTSL
Flowcode Product Page - Flowcode Help Wiki - Flowcode Examples - Flowcode Blog - Flowcode Course - My YouTube Channel
Flowcode Product Page - Flowcode Help Wiki - Flowcode Examples - Flowcode Blog - Flowcode Course - My YouTube Channel
- AbhijitR
- Posts: 300
- Joined: Fri Nov 07, 2014 12:48 pm
- Location: Pune, India
- Has thanked: 283 times
- Been thanked: 80 times
Re: ESP8266, MQTT, ThingSpeak
Hello! Ben
Good morning
Thank you for your answer, I will make the necessary changes as you suggested and keep you posted with the result in some time.
Noted about the login details, I shared only if someone wanted to check/validate something from thingspeak to be mentioned in chart.
Thank you again.
Abhi
Good morning
Thank you for your answer, I will make the necessary changes as you suggested and keep you posted with the result in some time.
Noted about the login details, I shared only if someone wanted to check/validate something from thingspeak to be mentioned in chart.
Thank you again.
Abhi
- AbhijitR
- Posts: 300
- Joined: Fri Nov 07, 2014 12:48 pm
- Location: Pune, India
- Has thanked: 283 times
- Been thanked: 80 times
Re: ESP8266, MQTT, ThingSpeak
Hello! Ben
I added the username and password fields as you said and tried, unfortunately without success, the C0 LED is still OFF, one more thing to mention the username is of 28 characters instead of 20, but still it complied, could that be a problem?
What is the use of the API key, as i saw the use of that key in 2 more videos but those were using Arduino and not PIC microcontroller.
Can you please give some more suggestions? I am clueless for the moment what to do.
Thank you
Abhi
I added the username and password fields as you said and tried, unfortunately without success, the C0 LED is still OFF, one more thing to mention the username is of 28 characters instead of 20, but still it complied, could that be a problem?
What is the use of the API key, as i saw the use of that key in 2 more videos but those were using Arduino and not PIC microcontroller.
Can you please give some more suggestions? I am clueless for the moment what to do.
Thank you
Abhi
- AbhijitR
- Posts: 300
- Joined: Fri Nov 07, 2014 12:48 pm
- Location: Pune, India
- Has thanked: 283 times
- Been thanked: 80 times
Re: ESP8266, MQTT, ThingSpeak
Hi!
Following my last post and unsuccessfully trials, I thought of sharing the details of the ESP8266 board i am using, if that is the problem then i will arrange the correct board required for my project. Below is the link of my ESP8266 board, it is a NodeMCU board, now i doubt if this is the correct board i am using or i have to use only the ESP8266 board.
NodeMCU which i am using for the moment
https://robu.in/product/nodemcu-cp2102-board/
Or do i need to use the below component
https://robu.in/product/esp8266-serial- ... le-fl-m1s/
Thank you.
Abhi
Following my last post and unsuccessfully trials, I thought of sharing the details of the ESP8266 board i am using, if that is the problem then i will arrange the correct board required for my project. Below is the link of my ESP8266 board, it is a NodeMCU board, now i doubt if this is the correct board i am using or i have to use only the ESP8266 board.
NodeMCU which i am using for the moment
https://robu.in/product/nodemcu-cp2102-board/
Or do i need to use the below component
https://robu.in/product/esp8266-serial- ... le-fl-m1s/
Thank you.
Abhi
- Benj
- Matrix Staff
- Posts: 15312
- Joined: Mon Oct 16, 2006 10:48 am
- Location: Matrix TS Ltd
- Has thanked: 4803 times
- Been thanked: 4314 times
- Contact:
Re: ESP8266, MQTT, ThingSpeak
Hello Abhi,
Looks like a nice board, as it's connecting to the network it probably should be ok. Have you had to load the firmware on yourself to allow it to be a slave to the MCU instead of hosting it's own firmware?
The MQTT component supports 32 byte name, password and ID so this hopefully shouldn't be a problem.
Has any data been posted to the Thingspeak account? As I say there is no ack from Thingspeak so your firmware will always report that the data has not been acknowledged.
API key might be for things like apps to connect but not too sure about this. Google says this.
Looks like a nice board, as it's connecting to the network it probably should be ok. Have you had to load the firmware on yourself to allow it to be a slave to the MCU instead of hosting it's own firmware?
The MQTT component supports 32 byte name, password and ID so this hopefully shouldn't be a problem.
Has any data been posted to the Thingspeak account? As I say there is no ack from Thingspeak so your firmware will always report that the data has not been acknowledged.
API key might be for things like apps to connect but not too sure about this. Google says this.
Presumably you have created the topic already in the ThingSpeak account settings.The User API Key is necessary to create and delete channels and to view information for private channels.
Regards Ben Rowland - MatrixTSL
Flowcode Product Page - Flowcode Help Wiki - Flowcode Examples - Flowcode Blog - Flowcode Course - My YouTube Channel
Flowcode Product Page - Flowcode Help Wiki - Flowcode Examples - Flowcode Blog - Flowcode Course - My YouTube Channel
- AbhijitR
- Posts: 300
- Joined: Fri Nov 07, 2014 12:48 pm
- Location: Pune, India
- Has thanked: 283 times
- Been thanked: 80 times
Re: ESP8266, MQTT, ThingSpeak
Hi! Ben
good evening
https://youtu.be/Kdg2Sr-bFZ8
Thank you again.
Abhi
good evening
Yes indeed it is a good board, and as you said what makes me feel confidant about the ESP8266 board is it is initializing well and also connecting to my mobile hotspot, just for one more confirmation i punched wrong password in the chart and it did not work. Coming to the firmware, i did not do anything just made the connections and powered up.Looks like a nice board, as it's connecting to the network it probably should be ok. Have you had to load the firmware on yourself to allow it to be a slave to the MCU instead of hosting it's own firmware?
You are right so i did not receive any error while compiling.The MQTT component supports 32 byte name, password and ID so this hopefully shouldn't be a problem.
No Ben unfortunately i do not see anything receiving on the webpage of Thingspeak, kindly excuse to speak my mind but i do not understand if i have entered correct details in the Thingspeak as well as in the chart which will complement each other to exchange the data, i think this could be the problem.Has any data been posted to the Thingspeak account? As I say there is no ack from Thingspeak so your firmware will always report that the data has not been acknowledged.
Regarding the API key i am sharing one video which has got all the things which i am looking for except PIC microcontrollerAPI key might be for things like apps to connect but not too sure about this. Google says this.
https://youtu.be/Kdg2Sr-bFZ8
Ben to make things little easy i am PMing you the log in details of thingspeak account, could you please check, please, no doubt as per your free time, i know i am asking more but it will be a very good start for me to explore IOT project.Presumably you have created the topic already in the ThingSpeak account settings.
Thank you again.
Abhi
-
- Valued Contributor
- Posts: 784
- Joined: Fri Jun 06, 2014 3:53 pm
- Has thanked: 186 times
- Been thanked: 205 times
Re: ESP8266, MQTT, ThingSpeak
Hi Abhi
Last year I used an ESP8266 to update Thingspeak whilst refamiliaring myself on one of your topics
viewtopic.php?p=106883#p106883
I had issues with the component macro ClientSendRequest so reverted to using the UART and AT commands. I've had no issue using the UART to interface with the ESP8266.
I'll see if I can dig out my tests, but if not it is easy enough to create another to have a PIC/ESP8266 directly update TS. Can't remember offhand if I ever updated TS using MQTT.
Regards
Edit
I also used ESP8266's here. Whilst not connecting to TS it will illustrate connecting and sending to another ESP.
https://www.flowcode.co.uk/forums/viewt ... 289#p10289
Last year I used an ESP8266 to update Thingspeak whilst refamiliaring myself on one of your topics
viewtopic.php?p=106883#p106883
I had issues with the component macro ClientSendRequest so reverted to using the UART and AT commands. I've had no issue using the UART to interface with the ESP8266.
I'll see if I can dig out my tests, but if not it is easy enough to create another to have a PIC/ESP8266 directly update TS. Can't remember offhand if I ever updated TS using MQTT.
Regards
Edit
I also used ESP8266's here. Whilst not connecting to TS it will illustrate connecting and sending to another ESP.
https://www.flowcode.co.uk/forums/viewt ... 289#p10289
-
- Valued Contributor
- Posts: 784
- Joined: Fri Jun 06, 2014 3:53 pm
- Has thanked: 186 times
- Been thanked: 205 times
Re: ESP8266, MQTT, ThingSpeak
Hi Abhi
I found a rough copy of my chart that updates TS using a PIC and the ESP8266. It is quite simple and was to test if I could connect and if so update TS with some data.
In it I see I did indeed revert to some AT commands via the UART to send the string required to update, as at the time the WLAN_ESP8266:ClientSendRequest didn't work.
If you want a copy let me know and I'll tidy it up and post.
Regards
I found a rough copy of my chart that updates TS using a PIC and the ESP8266. It is quite simple and was to test if I could connect and if so update TS with some data.
In it I see I did indeed revert to some AT commands via the UART to send the string required to update, as at the time the WLAN_ESP8266:ClientSendRequest didn't work.
If you want a copy let me know and I'll tidy it up and post.
Regards
- AbhijitR
- Posts: 300
- Joined: Fri Nov 07, 2014 12:48 pm
- Location: Pune, India
- Has thanked: 283 times
- Been thanked: 80 times
Re: ESP8266, MQTT, ThingSpeak
Hello! Chipfryer
good evening
First of all thank you for your response
May i bother you to for the example.
Thank you again.
Abhi
good evening
First of all thank you for your response
Yes i do remember about the above post, i tried to read and understand that but ..............Last year I used an ESP8266 to update Thingspeak whilst refamiliaring myself on one of your topics
Probably you are right but honestly i am not the correct person to make any judgement about the component as i am not sure what is going wrong in my case.I had issues with the component macro ClientSendRequest so reverted to using the UART and AT commands. I've had no issue using the UART to interface with the ESP8266.
May i bother you to for the example.
Thank you again.
Abhi
-
- Valued Contributor
- Posts: 784
- Joined: Fri Jun 06, 2014 3:53 pm
- Has thanked: 186 times
- Been thanked: 205 times
Re: ESP8266, MQTT, ThingSpeak
Hi
Attached is a rough copy of updating ThingSpeak using api keys.
When you create a channel on TS you will be given a unique api key and this is used to ensure any data you send goes to your channel.
The chart isn't very elegant and its purpose is just to show you the strings to update your channel, but you should consider including checks to see if you are connected and such like. We just assume all is good but you should check...
The chart will connect to your WiFi, then connect to TS and send data. Free version of TS will no doubt have limits on how many channels you can have and how frequently you can update. Don't leave a test running forever...... Make sure you are within your terms of service. This will loop sending data every 15 seconds. Also appreciate that the internet is not instant communications, latency has an effect. Your PIC (or whatever) can run far faster than infrastructure can handle.
Creating the strings are simple enough but you need to be careful to format correctly.
In the chart you will need to preassign certain variables listed below. Most are self explanatory. Some strings are at default size, others are at a length that will work for the test but may need altered to suit your application.
SSID = enter your WiFi SSID details
Password = enter your WiFi password details
Key = enter your API key details
Field = enter the field number of the field you wish to update (you can have many fields per channel) or populate dynamically
Data = your data to be sent. In the example I have this as a simple byte that increments each iteration
These next fields are partially populated and will be updated as we create strings etc as we progress:-
Cipsend = this is an AT command we will modify
StrSend = this is the actual string we send containing your key, field and data. We will modify as we progress
The WLAN component has a "bug" with ClientSendRequest or at least I cannot get it to work (always returning "no tail") so when we send the data we use the SendString to issue AT commands instead.
Incidentally when working with components such as the ESP, having a very basic USB-TTL serial converter is invaluable as you can use it to see what is being sent to/from the ESP to/from your PIC. It also allows you to send AT commands directly to the ESP from your PC and see any reply..
The chart initialises, connects to your WiFi and then connect to TS. We then branch to a User Macro (Get_Data). In here you can put your code to gather whatever data you wish. Once collected we need to process it a bit further. Unless you are gathering string data, we need to convert your number (byte/integer/float/whatever) to a string.
Returning from that Macro we then construct the string we will send (StrSend) by adding in your key, field and data etc. Once done we need to then obtain the length of the string (Length) and then convert this value to a string (LenStr). We need to do this as we have to not only tell the server we are going to send something but also how many bytes to expect.
We issue the command AT+CIPSEND informing of how many bytes we are sending then we pause for a few hundred milliseconds to allow time for a response before we continue. I'm using 400mS here but at home I'm good with 200mS or less. Of course you could modify to look for a ">" response instead before proceeding.
We then send the string containing your api key, field and data. This should update the field in your TS Channel.
Clearly there is more work to do to turn this into something more useful but does illustrate how to use the API key. It might be worth taking the time to ensure your ESP8266 is running the latest firmware/AT as I've bought ones that were years behind. My ESP's remember the last SSID they connected to and upon boot they automatically attempt to connect. You may want to include a check to see if you are already connected before running code to connect etc.
I did create a chart to update TS using MQTT but not sure where that is or how successful I was. If I find it I'll let you know.
Hope this helps.
Regards
EDIT
In my Evil Lab I connect via the Cell network. Fastest I could reliably update free TS was with a delay between updates of ~12s.
Attached is a rough copy of updating ThingSpeak using api keys.
When you create a channel on TS you will be given a unique api key and this is used to ensure any data you send goes to your channel.
The chart isn't very elegant and its purpose is just to show you the strings to update your channel, but you should consider including checks to see if you are connected and such like. We just assume all is good but you should check...
The chart will connect to your WiFi, then connect to TS and send data. Free version of TS will no doubt have limits on how many channels you can have and how frequently you can update. Don't leave a test running forever...... Make sure you are within your terms of service. This will loop sending data every 15 seconds. Also appreciate that the internet is not instant communications, latency has an effect. Your PIC (or whatever) can run far faster than infrastructure can handle.
Creating the strings are simple enough but you need to be careful to format correctly.
In the chart you will need to preassign certain variables listed below. Most are self explanatory. Some strings are at default size, others are at a length that will work for the test but may need altered to suit your application.
SSID = enter your WiFi SSID details
Password = enter your WiFi password details
Key = enter your API key details
Field = enter the field number of the field you wish to update (you can have many fields per channel) or populate dynamically
Data = your data to be sent. In the example I have this as a simple byte that increments each iteration
These next fields are partially populated and will be updated as we create strings etc as we progress:-
Cipsend = this is an AT command we will modify
StrSend = this is the actual string we send containing your key, field and data. We will modify as we progress
The WLAN component has a "bug" with ClientSendRequest or at least I cannot get it to work (always returning "no tail") so when we send the data we use the SendString to issue AT commands instead.
Incidentally when working with components such as the ESP, having a very basic USB-TTL serial converter is invaluable as you can use it to see what is being sent to/from the ESP to/from your PIC. It also allows you to send AT commands directly to the ESP from your PC and see any reply..
The chart initialises, connects to your WiFi and then connect to TS. We then branch to a User Macro (Get_Data). In here you can put your code to gather whatever data you wish. Once collected we need to process it a bit further. Unless you are gathering string data, we need to convert your number (byte/integer/float/whatever) to a string.
Returning from that Macro we then construct the string we will send (StrSend) by adding in your key, field and data etc. Once done we need to then obtain the length of the string (Length) and then convert this value to a string (LenStr). We need to do this as we have to not only tell the server we are going to send something but also how many bytes to expect.
We issue the command AT+CIPSEND informing of how many bytes we are sending then we pause for a few hundred milliseconds to allow time for a response before we continue. I'm using 400mS here but at home I'm good with 200mS or less. Of course you could modify to look for a ">" response instead before proceeding.
We then send the string containing your api key, field and data. This should update the field in your TS Channel.
Clearly there is more work to do to turn this into something more useful but does illustrate how to use the API key. It might be worth taking the time to ensure your ESP8266 is running the latest firmware/AT as I've bought ones that were years behind. My ESP's remember the last SSID they connected to and upon boot they automatically attempt to connect. You may want to include a check to see if you are already connected before running code to connect etc.
I did create a chart to update TS using MQTT but not sure where that is or how successful I was. If I find it I'll let you know.
Hope this helps.
Regards
EDIT
In my Evil Lab I connect via the Cell network. Fastest I could reliably update free TS was with a delay between updates of ~12s.
- AbhijitR
- Posts: 300
- Joined: Fri Nov 07, 2014 12:48 pm
- Location: Pune, India
- Has thanked: 283 times
- Been thanked: 80 times
Re: ESP8266, MQTT, ThingSpeak
Hello! Chipfryer
good morning
Thank you once again for your efforts, I agree with you the component did not worked unfortunately for me too, or may be i did something wrong, but no corrections.
Your chart and the explanation you gave are exceptional (as always), thank you again for those details (the devil is in the details )
As you mentioned i am adding the confirmation branches at every step to make sure all the connections are good, i also noticed one more thing at any time in the main loop if the connection is lost, need to check if it is available again, and many such actions, big homework.
But i am happy to see the chart is working and populating the TS.
Cheers.....
Abhi
good morning
Thank you once again for your efforts, I agree with you the component did not worked unfortunately for me too, or may be i did something wrong, but no corrections.
Your chart and the explanation you gave are exceptional (as always), thank you again for those details (the devil is in the details )
As you mentioned i am adding the confirmation branches at every step to make sure all the connections are good, i also noticed one more thing at any time in the main loop if the connection is lost, need to check if it is available again, and many such actions, big homework.
But i am happy to see the chart is working and populating the TS.
Cheers.....
Abhi
-
- Valued Contributor
- Posts: 784
- Joined: Fri Jun 06, 2014 3:53 pm
- Has thanked: 186 times
- Been thanked: 205 times
Re: ESP8266, MQTT, ThingSpeak
Hi Abhi
Glad to hear it is working for you and you can update.
By no means was my chart a "finished product", many checks should be incorporated, but it let me construct a frame to build around. The bug in the ESP8266 is I believe due to Espressif updating firmware after the component was created and subsequently changing the reply the component is expecting.
Download a copy of the AT commands and you should be able to do pretty much anything with it. The little module is quite handy.
Regards
Glad to hear it is working for you and you can update.
By no means was my chart a "finished product", many checks should be incorporated, but it let me construct a frame to build around. The bug in the ESP8266 is I believe due to Espressif updating firmware after the component was created and subsequently changing the reply the component is expecting.
Download a copy of the AT commands and you should be able to do pretty much anything with it. The little module is quite handy.
Regards