DATA TO .CSV FILE

[chipfryer27]

Hi

Following on, attached is a simple chart to capture Temperature and Humidity using a DHT11, then send the data to Excel.

I had a typo in my previous post regarding End of Packet. I meant to say that this is Line Feed (\n) not Carriage Return.

The chart initialises then loops every ten seconds. In the loop we sample the sensor for two values, temperature and humidity. If the sensor is unavailable we send out a preformatted CSV string ending with Line Feed (N/A,N/A\n).

If available we send the readings out individually, inserting the necessary comma and LF character.

There are many ways to gather sample data and create CSV's to send, however this is just a very simple example that will send data to Excel DataStreamer. You can also send text, not just numbers. Modify to suit.

Excel Test v10_EB2.fcfx

(19.93 KiB) Downloaded 332 times

Regards

[SILVESTROS]

Hi chipfryer27,
many thanks for info and help...I'll test the code for data to excel. I would like to have your opinion about the following . I try to get data from 2 sensors and send that with a string to ESP8266..sensors are communicate with mcu with I2C..in the following demo code are there mistakes ? I get errors in simulation and the code not compile... also how can send sensors data variables in one string to ESP8266? after that, the string will be send with WIFI to a second ESP, that is connected with PC with serial/USB...sensors data are record in Excel..

[chipfryer27]

Hi

Sorry to be brief

I've had a very, very quick look and it looks like you are running using a 4MHz external oscillator and using the 4xPLL to bring this up to 16MHz? Also I see you have the Watchdog Timer enabled but may have forgotten to include clears as I don't see any. This would mean the chip would reset itself approx every 2.18 minutes.

I'm not sure why you are using connection points as I don't immediately see why a Loop would not suffice as you are essentially creating such. I see too that in each iteration, you are initialising all components. This is unnecessary as they only need to be initialised once and therefore you could put them early in the chart before you "loop".

I'll have a proper look later but in the meantime have you looked at the ESP8266 examples in the WiKi ? I think the last time I used the ESP8266 I didn't use the component, instead I sent "AT" commands via the UART. I'll have a check later.

Regards

[chipfryer27]

Hi

I've had a further look.

First I'd like to say that I haven't yet used the Accelerometer nor Oximeter components myself, so the following is a "best guess" from WiKi / simulation

As mentioned earlier, all initialising settings can be at the beginning and this could include the ESP connecting to WiFi / server too. You don't mention how many samples per second/minute/hour you need, so I assume you will gather as fast as you can? I won't comment on the ESP settings yet as I need to refresh my memory as it was quite a few months since I last used, and I think I just used the UART.

The Accelerometer returns 12-bit readings for x/y/z however you forgot to assign an integer variable, only the default byte, which can only deal with 8-bit values. This would cause incorrect readings. Similarly, the Oximeter needs to deal with 16-bit numbers but again only a default Byte has been assigned. Incidentally it's good practise to assign an initial value to each variable so it always starts at a known value.

Although you initialised both these components, you forgot to include the associated "update" command, which actually obtains all data from sensor(s), before you collect/read samples. Easy to overlook.

Although not necessary for simulation, I see that your set IR/Red values are a bit low, although I stress I am only going on what I've read. I think the IR needs 50mA which equates to 15 and Red needs around 27mA which I guess equates to around 8.

I can't really comment on pulse width nor sampling rate as I haven't used the components, but I guess 20mS and 50Hz are a good match.

I've modified your chart config and to have all initialising at the beginning but I've "disabled" all ESP associated commands as I haven't looked into them yet. If you run the chart in simulation (either Fast with Updates or just Step-Thru) with the Simulation Debugger set to show your x/y/x and hrate/oxm variables, you will see them change as you adjust the respective sliders. Note the accelerometer component only shows integer values of 0/1/2 but the returned readings show the full 12-bit value. A little bug I think

As to creating a string to send, you first need to establish how the receiver demands the data as you must create a string the receiver can read. If you can let us know what format we can help with creation, but it should be an easy enough string calculation.

As an aside, I was playing with Excel DataStreamer with mixed results. Although as documented in an earlier post I can send data and it can accept and display, when it saves the data it somehow screws it up. The CSV formatting gets corrupted so instead of, for example

13.03,24.14(\n)13.04,24.15(\n)

It saves as 13.03,24.14.13,04.24.15 and the data is imported as a very long single row....

Possibly a problem in Excel that an update / patch may fix, still looking into it but it's not a big priority. If I am connecting directly to a PC then I would most likely use Flowcode App Developer as it allows me far greater control over everything, and if I was just wishing to capture data as in your example and I had an Internet connection, I would probably send to ThingSpeak or such like as that eliminates the need for a PC entirely. Data can be read/ viewed online and downloaded.

I hope the attached helps and I'll look at the ESP8266 part later

Regards

PIC18F23K22-SENSORS-DATA1_Mod_1.fcfx

(21.56 KiB) Downloaded 399 times

EDIT...
Just read your other post re compilation. The above won't compile either for similar reasons but does illustrate how to read the sensors.

[mnfisher]

Thanks Iain,

Just as an idea - does using \r\n as end of line work?

Martin

[chipfryer27]

Hi Martin

Just read your post as I was about to update and clearly my intelligence is improving from reading all your posts as I had just done as you suggested

Excel documentation made no mention of CR, only LF which is why initially I created the chart with only LF. This worked in that Excel would capture the data and display, but when saving and trying to reopen the problems began.

The CSV file was one long string (as viewed in Notepad or Import Preview) and was incorrectly formatted as documented above. In addition, if trying to Import Data File from within Excel DS nothing happened. A few screen flashes as if something was happening but no data.

Adding CR to now make the string \r\n didn't affect the gathering of the data. Excel DS happily accepted and displayed. Now however when the files was saved it was as hoped with multiple rows of data in the correct format, however the timestamp isn't included.

Headers blah blah blah
12.34,56.78
12.56,78.32

Incidentally whilst in simulation the DHT11 displays both temperature and humidity to two decimal places, it only prints / sends humidity as an integer.

Once saved, in DS I could then select Import Data File then Play Data which then populated the chart including timestamp. Do note that DS only allows 500 rows so there is a limit to how much you can view with a timestamp.

I can see this being useful for some applications, but as mentioned above FAD / ThingSpeak / etc for me offer more.

Regards

[SILVESTROS]

many thanks for your time and help...I'm waiting for some chips with larger memory ( PIC18F25K22) to make some tests...this chip is to control second ESP8266 ...my idea is to use 2 ESP8266 connected with wifi to transfer sensors data to PC and record that in Excel...first ESP8266 is connected with a PIC18F46K22/44QFP, to collect sensors data and send that with wifi to second ESP, that is connected with a PIC18F25K22...I select that because it has 2 uarts ...so, uart1 is connected with ESP, uart2 is connected with serial/USB module to send data to PC and then to Excel with DS..sensors data are displayed on data sheet in real time...for now I'll make some tests with simulation...

[chipfryer27]

Hi

So from my understanding you are trying to create a transparent link between the PIC with sensors and a PIC connected to your PC, using Excel DS to capture. Whatever "sensor PIC" sends, the PC will receive.

If both ESPs are connecting via a router, then you will most likely need to give a fixed IP to your ESP8266/PIC connected to the PC, but I'm unsure if there is a component to assign an IP address so you might need to resort to some "AT" commands.

All sounds very doable though.

Regards