Size limitation on look up tables?

[Tortilla]

I would like to use a look up table in V9 to store 8840 hex values separated by commas. This is in the format:

0x15, 0x00, 0x03, 0x74, 0x0B, 0xD4, 0x84, 0x60,
0x16, 0x6F, 0xAE, 0x6C, 0xF9, 0xBB, 0x84, 0xA2,

. . . . . .

I will need to load all 8840 hex values in their original order as a patch over the I2C bus to another device.

I created a look up table, but it was only able to store 4616 values. It would not allow me to add the additional 4224 values.

I was able to create another look up table and store the remaining 4224 values. However, I would prefer to only use one look up table rather than having to use two look up tables.

Is there a size limitation on individual look up tables? Is there some way I could use one look up table to store all the 8840 values?

Thank you.

[mnfisher]

Try using an array and set the initial value.

Might be a job for the embedded file component (although this gets very slow for large files ~8k should be okay)

For large blocks of data - I've used a little C code with success (#include "data.dat" where data.dat file is MX_UINT8 data[] = {0x12, 0x23, ..}; - the values can be embedded into program memory (rather than RAM) using PROGMEM if required - I've used this for ~250k of graphics on esp32....)

Martin

[canary_wharfe]

Hello Martin

That's a really good workaround and a great idea. Wasn't aware you could use PROGMEM for such large amount of data in Flowcode. Any chance of posting a small demo flowcode illustrating the important code blocks and showing the C code syntax that would represent a reference for inexperienced flowcoders that want to replicate your suggestion?

[mnfisher]

This is how I did it for ESP32 (I think the syntax is different for adding data to program memory on different MCUs) - I looked at the code generated by an embedded file to see how to do it.

File is included in project options->supplementary code. The data file needs to be in project\main\ (again for esp32) or you can specify a path in the #include statement.

I've demonstrated sending all 26 bytes of my 'demo' file in a single call. It is a bit silly here - and UART code is subject to name mangling which makes life a little harder (If it won't compile look at PrintData) - however the technique can be used for TransactionWrite for SPI etc (and the names of the functions don't get mangled)

To use large files on esp32 you might also need to alter the partition table.

data.fcfx

(11.92 KiB) Downloaded 356 times

data.zip

(224 Bytes) Downloaded 274 times

Martin

[mnfisher]

A simple Python script to convert 'data' - here it appends '.bin' to the filename - and outputs as '.dat'

Code: Select all

#!python3
import binascii
import sys

for fn in sys.argv[1:]:
    print(fn + ".bin -> " + fn + ".dat")
    with open(fn + ".dat", "w") as out:
        out.write("ROMARRAY_(MX_UINT8) " + fn + " ROMARRAY_E = {")
        pos = 0
        with open(fn + ".bin", "rb") as f:
            data = binascii.hexlify(f.read()).upper()
            for i in range(0, len(data) - 1, 2):            
                str = "0x" + chr(data[i])  + chr(data[i+1])
                out.write(str)
                if i < len(data) - 2 :
                    out.write(",")
                    pos += 1
                    if pos == 30:
                        pos = 0 
                        out.write("\r\n")
            out.write("};\r\n\n")
print("End")

So

tohex.py data

converts data.bin to data.dat (name of the array is 'data' in this case)

Data.dat can be included into FC using #include...

(Note it will convert multiple files

tohex.py d1 d2 d3

will convert d1.bin, d2.bin and d3.bin (arrays called d1, d2 and d3

Martin

[Tortilla]

Thank you to all who responded. Great ideas and I appreciate it.

Martin, I also thought about using the "embedded file" component--I will have to add the storage feature to my v9 subscription, but it might be the least effort overall.

Speed is not an issue for me--I can wait for the patch to load over the I2C bus. So long as it could be stored in one file, I would be good to go.

I can try creating an array, but I wonder it it will have the same issues that the lookup table has since lookup tables are just arrays (as I understand it)? Of course, I just need to try it. I won't be able to get back to the project today.

I'm using a PIC 18LF2685 which has 96K of flash memory.

Thanks.