ESP8266, MQTT, ThingSpeak

For Flowcode users to discuss projects, flowcharts, and any other issues related to Flowcode 8.

Moderator: Benj

Post Reply
User avatar
AbhijitR
Posts: 300
Joined: Fri Nov 07, 2014 12:48 pm
Location: Pune, India
Has thanked: 283 times
Been thanked: 80 times

ESP8266, MQTT, ThingSpeak

Post by AbhijitR »

Hello!

Since last couple of days i am working on one of the projects from MQTT Client (Component)

https://www.matrixtsl.com/wiki/images/c ... P8266.fcfx

I made the necessary changes in the chart as i wanted, on switching on the hardware i noticed the wifi module is getting connected to my mobile hotspot (i can see that in the connected devices in my phone), so one thing is sure the chart is working well until getting in the loop or initializing is done correctly, the SSID macro is working correctly as the name and password working good for the hotspot.

But further the Connected decision is not working as i do not see the LED added to the C0 port getting ON in that branch, what probable mistake i am doing can someone help. Did i miss any properties to be punched in for any component, i have also created an account on thingspeak to see the analog value but nothing is displayed on the plot.

No doubt i am doing some mistake but i did not find any help on google (Flowcode+Thingspeak+MQTT) nor forum, or probably i missed the correct words while searching, this is new for me to understand. In one of the videos i learned it is needed to punch the API key but again where in Flowcode i did not understand, can someone give me some hint or spare some time to check my chart, it will be a great help.

Thank you.

Abhi
Attachments
IOT_Project_18F27K42_25-05-2023.fcfx
(16.05 KiB) Downloaded 279 times

User avatar
Benj
Matrix Staff
Posts: 15312
Joined: Mon Oct 16, 2006 10:48 am
Location: Matrix TS Ltd
Has thanked: 4803 times
Been thanked: 4314 times
Contact:

Re: ESP8266, MQTT, ThingSpeak

Post by Benj »

Hello Abhi,

Investigating for you now.

Please do not share login details.

User avatar
Benj
Matrix Staff
Posts: 15312
Joined: Mon Oct 16, 2006 10:48 am
Location: Matrix TS Ltd
Has thanked: 4803 times
Been thanked: 4314 times
Contact:

Re: ESP8266, MQTT, ThingSpeak

Post by Benj »

In the MQTT Client component you maybe need to turn the Authentication property to On and then enter your thingspeak username and password.

This is also interesting.
ThingSpeak implements MQTT publish and subscribe functionality with QoS = 0. This "fire-and-forget" protocol means that your client does not receive an acknowledgment when you publish to a ThingSpeak channel.
This means when you publish daat you won't get an acknowledge signal meaning your program will go into disconnected mode and then re-connect. This when working correctly should not be required.

User avatar
AbhijitR
Posts: 300
Joined: Fri Nov 07, 2014 12:48 pm
Location: Pune, India
Has thanked: 283 times
Been thanked: 80 times

Re: ESP8266, MQTT, ThingSpeak

Post by AbhijitR »

Hello! Ben
Good morning

Thank you for your answer, I will make the necessary changes as you suggested and keep you posted with the result in some time.

Noted about the login details, I shared only if someone wanted to check/validate something from thingspeak to be mentioned in chart.

Thank you again.

Abhi

User avatar
AbhijitR
Posts: 300
Joined: Fri Nov 07, 2014 12:48 pm
Location: Pune, India
Has thanked: 283 times
Been thanked: 80 times

Re: ESP8266, MQTT, ThingSpeak

Post by AbhijitR »

Hello! Ben

I added the username and password fields as you said and tried, unfortunately without success, the C0 LED is still OFF, :( one more thing to mention the username is of 28 characters instead of 20, but still it complied, could that be a problem?

What is the use of the API key, as i saw the use of that key in 2 more videos but those were using Arduino and not PIC microcontroller.

Can you please give some more suggestions? I am clueless for the moment what to do.

Thank you

Abhi

User avatar
AbhijitR
Posts: 300
Joined: Fri Nov 07, 2014 12:48 pm
Location: Pune, India
Has thanked: 283 times
Been thanked: 80 times

Re: ESP8266, MQTT, ThingSpeak

Post by AbhijitR »

Hi!

Following my last post and unsuccessfully trials, I thought of sharing the details of the ESP8266 board i am using, if that is the problem then i will arrange the correct board required for my project. Below is the link of my ESP8266 board, it is a NodeMCU board, now i doubt if this is the correct board i am using or i have to use only the ESP8266 board.

NodeMCU which i am using for the moment
https://robu.in/product/nodemcu-cp2102-board/

Or do i need to use the below component
https://robu.in/product/esp8266-serial- ... le-fl-m1s/

Thank you.

Abhi

User avatar
Benj
Matrix Staff
Posts: 15312
Joined: Mon Oct 16, 2006 10:48 am
Location: Matrix TS Ltd
Has thanked: 4803 times
Been thanked: 4314 times
Contact:

Re: ESP8266, MQTT, ThingSpeak

Post by Benj »

Hello Abhi,

Looks like a nice board, as it's connecting to the network it probably should be ok. Have you had to load the firmware on yourself to allow it to be a slave to the MCU instead of hosting it's own firmware?

The MQTT component supports 32 byte name, password and ID so this hopefully shouldn't be a problem.

Has any data been posted to the Thingspeak account? As I say there is no ack from Thingspeak so your firmware will always report that the data has not been acknowledged.

API key might be for things like apps to connect but not too sure about this. Google says this.
The User API Key is necessary to create and delete channels and to view information for private channels.
Presumably you have created the topic already in the ThingSpeak account settings.

User avatar
AbhijitR
Posts: 300
Joined: Fri Nov 07, 2014 12:48 pm
Location: Pune, India
Has thanked: 283 times
Been thanked: 80 times

Re: ESP8266, MQTT, ThingSpeak

Post by AbhijitR »

Hi! Ben
good evening
Looks like a nice board, as it's connecting to the network it probably should be ok. Have you had to load the firmware on yourself to allow it to be a slave to the MCU instead of hosting it's own firmware?
Yes indeed it is a good board, and as you said what makes me feel confidant about the ESP8266 board is it is initializing well and also connecting to my mobile hotspot, just for one more confirmation i punched wrong password in the chart and it did not work. Coming to the firmware, i did not do anything just made the connections and powered up.
The MQTT component supports 32 byte name, password and ID so this hopefully shouldn't be a problem.
You are right so i did not receive any error while compiling.
Has any data been posted to the Thingspeak account? As I say there is no ack from Thingspeak so your firmware will always report that the data has not been acknowledged.
No Ben unfortunately i do not see anything receiving on the webpage of Thingspeak, kindly excuse to speak my mind but i do not understand if i have entered correct details in the Thingspeak as well as in the chart which will complement each other to exchange the data, i think this could be the problem.
API key might be for things like apps to connect but not too sure about this. Google says this.
Regarding the API key i am sharing one video which has got all the things which i am looking for except PIC microcontroller :)
https://youtu.be/Kdg2Sr-bFZ8
Presumably you have created the topic already in the ThingSpeak account settings.
Ben to make things little easy i am PMing you the log in details of thingspeak account, could you please check, please, no doubt as per your free time, i know i am asking more but it will be a very good start for me to explore IOT project.

Thank you again.

Abhi

chipfryer27
Valued Contributor
Valued Contributor
Posts: 784
Joined: Fri Jun 06, 2014 3:53 pm
Has thanked: 186 times
Been thanked: 205 times

Re: ESP8266, MQTT, ThingSpeak

Post by chipfryer27 »

Hi Abhi

Last year I used an ESP8266 to update Thingspeak whilst refamiliaring myself on one of your topics :)

viewtopic.php?p=106883#p106883

I had issues with the component macro ClientSendRequest so reverted to using the UART and AT commands. I've had no issue using the UART to interface with the ESP8266.

I'll see if I can dig out my tests, but if not it is easy enough to create another to have a PIC/ESP8266 directly update TS. Can't remember offhand if I ever updated TS using MQTT.

Regards

Edit
I also used ESP8266's here. Whilst not connecting to TS it will illustrate connecting and sending to another ESP.
https://www.flowcode.co.uk/forums/viewt ... 289#p10289

chipfryer27
Valued Contributor
Valued Contributor
Posts: 784
Joined: Fri Jun 06, 2014 3:53 pm
Has thanked: 186 times
Been thanked: 205 times

Re: ESP8266, MQTT, ThingSpeak

Post by chipfryer27 »

Hi Abhi

I found a rough copy of my chart that updates TS using a PIC and the ESP8266. It is quite simple and was to test if I could connect and if so update TS with some data.

In it I see I did indeed revert to some AT commands via the UART to send the string required to update, as at the time the WLAN_ESP8266:ClientSendRequest didn't work.

If you want a copy let me know and I'll tidy it up and post.

Regards

User avatar
AbhijitR
Posts: 300
Joined: Fri Nov 07, 2014 12:48 pm
Location: Pune, India
Has thanked: 283 times
Been thanked: 80 times

Re: ESP8266, MQTT, ThingSpeak

Post by AbhijitR »

Hello! Chipfryer
good evening

First of all thank you for your response
Last year I used an ESP8266 to update Thingspeak whilst refamiliaring myself on one of your topics :)
Yes i do remember about the above post, i tried to read and understand that but .............. :?:
I had issues with the component macro ClientSendRequest so reverted to using the UART and AT commands. I've had no issue using the UART to interface with the ESP8266.
Probably you are right but honestly i am not the correct person to make any judgement about the component as i am not sure what is going wrong in my case.

May i bother you to for the example.

Thank you again.

Abhi

chipfryer27
Valued Contributor
Valued Contributor
Posts: 784
Joined: Fri Jun 06, 2014 3:53 pm
Has thanked: 186 times
Been thanked: 205 times

Re: ESP8266, MQTT, ThingSpeak

Post by chipfryer27 »

Hi

Attached is a rough copy of updating ThingSpeak using api keys.

ThingSpeak-v8_Example_WlanESP8266-v1.fcfx
(19.48 KiB) Downloaded 303 times


When you create a channel on TS you will be given a unique api key and this is used to ensure any data you send goes to your channel.

The chart isn't very elegant and its purpose is just to show you the strings to update your channel, but you should consider including checks to see if you are connected and such like. We just assume all is good but you should check...

The chart will connect to your WiFi, then connect to TS and send data. Free version of TS will no doubt have limits on how many channels you can have and how frequently you can update. Don't leave a test running forever...... Make sure you are within your terms of service. This will loop sending data every 15 seconds. Also appreciate that the internet is not instant communications, latency has an effect. Your PIC (or whatever) can run far faster than infrastructure can handle.

Creating the strings are simple enough but you need to be careful to format correctly.

In the chart you will need to preassign certain variables listed below. Most are self explanatory. Some strings are at default size, others are at a length that will work for the test but may need altered to suit your application.

SSID = enter your WiFi SSID details
Password = enter your WiFi password details
Key = enter your API key details
Field = enter the field number of the field you wish to update (you can have many fields per channel) or populate dynamically
Data = your data to be sent. In the example I have this as a simple byte that increments each iteration

These next fields are partially populated and will be updated as we create strings etc as we progress:-

Cipsend = this is an AT command we will modify
StrSend = this is the actual string we send containing your key, field and data. We will modify as we progress

The WLAN component has a "bug" with ClientSendRequest or at least I cannot get it to work (always returning "no tail") so when we send the data we use the SendString to issue AT commands instead.

Incidentally when working with components such as the ESP, having a very basic USB-TTL serial converter is invaluable as you can use it to see what is being sent to/from the ESP to/from your PIC. It also allows you to send AT commands directly to the ESP from your PC and see any reply..

The chart initialises, connects to your WiFi and then connect to TS. We then branch to a User Macro (Get_Data). In here you can put your code to gather whatever data you wish. Once collected we need to process it a bit further. Unless you are gathering string data, we need to convert your number (byte/integer/float/whatever) to a string.

Returning from that Macro we then construct the string we will send (StrSend) by adding in your key, field and data etc. Once done we need to then obtain the length of the string (Length) and then convert this value to a string (LenStr). We need to do this as we have to not only tell the server we are going to send something but also how many bytes to expect.

We issue the command AT+CIPSEND informing of how many bytes we are sending then we pause for a few hundred milliseconds to allow time for a response before we continue. I'm using 400mS here but at home I'm good with 200mS or less. Of course you could modify to look for a ">" response instead before proceeding.

We then send the string containing your api key, field and data. This should update the field in your TS Channel.

Clearly there is more work to do to turn this into something more useful but does illustrate how to use the API key. It might be worth taking the time to ensure your ESP8266 is running the latest firmware/AT as I've bought ones that were years behind. My ESP's remember the last SSID they connected to and upon boot they automatically attempt to connect. You may want to include a check to see if you are already connected before running code to connect etc.

I did create a chart to update TS using MQTT but not sure where that is or how successful I was. If I find it I'll let you know.

Hope this helps.

Regards

EDIT
In my Evil Lab I connect via the Cell network. Fastest I could reliably update free TS was with a delay between updates of ~12s.

User avatar
AbhijitR
Posts: 300
Joined: Fri Nov 07, 2014 12:48 pm
Location: Pune, India
Has thanked: 283 times
Been thanked: 80 times

Re: ESP8266, MQTT, ThingSpeak

Post by AbhijitR »

Hello! Chipfryer
good morning

Thank you once again for your efforts, I agree with you the component did not worked unfortunately for me too, or may be i did something wrong, but no corrections.

Your chart and the explanation you gave are exceptional (as always), thank you again for those details (the devil is in the details :evil: )

As you mentioned i am adding the confirmation branches at every step to make sure all the connections are good, i also noticed one more thing at any time in the main loop if the connection is lost, need to check if it is available again, and many such actions, big homework.

But i am happy to see the chart is working and populating the TS.

Cheers.....

Abhi

chipfryer27
Valued Contributor
Valued Contributor
Posts: 784
Joined: Fri Jun 06, 2014 3:53 pm
Has thanked: 186 times
Been thanked: 205 times

Re: ESP8266, MQTT, ThingSpeak

Post by chipfryer27 »

Hi Abhi

Glad to hear it is working for you and you can update.

By no means was my chart a "finished product", many checks should be incorporated, but it let me construct a frame to build around. The bug in the ESP8266 is I believe due to Espressif updating firmware after the component was created and subsequently changing the reply the component is expecting.

Download a copy of the AT commands and you should be able to do pretty much anything with it. The little module is quite handy.

Regards

Post Reply