Alex Spataru
  • Home
  • Blog
  • Projects
    • Serial Studio
  • Home
  • Blog
  • Projects
    • Serial Studio
Search

Blog posts, ideas & rambles...

Introducing Serial Studio, a dashboard software for serial port projects

12/29/2020

19 Comments

 
Did you ever have the need to display data from a microcontroller on a dashboard, and spent more time developing (and fixing) your dashboard software, than actually working on your MCU project?

Well, I did, multiple times. Let me put you in context, I participate in several CanSat competition programs through KA'AN SAT, a representative team at my university. A CanSat is “a simulation of a real satellite, integrated within the volume and shape of a soft drink can” (European Space Agency, more info). One of the main tasks in these competitions is to develop software for the ground station. The ground station software (GSS) receives telemetry from the CanSat in real time through a serial device (generally a XBee), displays it and exports it to a CSV/Excel file for post-mission analysis. 

From the start, we developed the GSS with Qt to support multiple operating systems (in case that one of our computers experienced problems during the competition) and because Qt/QML is very convenient for developing eye-catching user interfaces.

Here is a screenshot of the 2019 ground station software, and a photo of the GSS running & displaying telemetry during the CUCEI CanSat competition:
If you are interested, the source code for the 2019 GSS is available here. The software worked quite well for all its intents & purposes (we got the first place after all).

The problems came half a year later, when I found myself working on multiple projects that required some kind of data acquisition with serial devices. For example, some members of ROCH (another representative team in our university, which participates in the NASA Human Rover Exploration Challenge) wanted to integrate our GSS with their rover as a side-project during quarantine.

The result was disastrous; I got a call late at night and we ended up pulling an all-nighter to come up with a way to adapt the GSS with the telemetry that they were receiving. Finally, the software worked, but the UI integration was horrible & data export was not really functional (see the screenshot below to get an idea):
Picture
Both projects (the CanSat and the rover) had a similar telemetry format (sensor readings & OBC status data separated with commas). However, the information itself had a different order.
 
After that experience, I decided that I had enough of writing separate dashboard software for every project that I got involved with. I needed to come up with a way to have the same dashboard/GSS software to work with all projects, without the need of modifying the GSS code at the last minute.
 
The initial solution that I came up with was to create a JSON-based communication protocol between the GSS and the microcontroller, and thus Serial Studio was born (its initial name was SigLAB, but I changed my mind later). 

​Basically, the microcontroller sends the following information through the serial port:
  • Project title.
  • Current sensor readings & OBC status.
  • What each reading meant, its measurement units and what should the GSS do with that reading (e.g. create a real-time graph of the measured atmospheric pressure).

All this information can be easily represented in a JSON document, for example:
{
   "t":"KAANSATQRO",
   "g":[
      {
         "t":"Mission Status",
         "d":[
            {
               "t":"Runtime",
               "v":"%value%",
               "u":"ms"
            },
            {
               "t":"Packet count",
               "v":"%value%"
            },
            {
               "t":"Battery voltage",
               "v":"%value%",
               "g":true,
               "u":"V"
            }
         ]
      },
      {
         "t":"Sensor Readings",
         "d":[
            {
               "t":"Temperature",
               "v":"%value%",
               "g":true,
               "u":"°C"
            },
            {
               "t":"Altitude",
               "v":"%value%",
               "u":"m"
            },
            {
               "t":"Pressure",
               "v":"%value%",
               "u":"KPa",
               "g":true
            },
            {
               "t":"External Temperature",
               "v":"%value%",
               "g":true,
               "u":"°C"
            },
            {
               "t":"Humidity",
               "v":"%9",
               "g":true,
               "u":"%value%"
            }
         ]
      }
   ]
}
​
​As you can see, we have the following structure:
  • Project title
  • Array of data groups
    • For each group:
      • Group title
      • Array of datasets
        • For each dataset:
          • Title
          • Value
          • Units
          • Graph request (or not)

A group consists of values that are closely related to each other, for example:
  • OBC status (first group in our example)
  • Sensor readings (second group in our example)
  • Accelerometer readings (X, Y, Z)
  • GPS readings
  • Etc.

​On the other hand, datasets represent what each individual value means, and what we should do with it.

On Serial Studio, this information is displayed in the following manner:
Picture
As you can probably deduce, each “window” corresponds to a group in our JSON document (I hid the graphs in the screenshot to avoid confusion). 

This approach works beautifully for small projects. However, for more complex projects, creating and sending a large JSON document through serial (or through radio signals, and then through a serial port) becomes quite problematic. The solution? Load the same JSON document from your computer, instruct the microcontroller to send ONLY the sensor/data readings & let Serial Studio figure out the rest by using the indices of each received value in a comma-separated data frame.

Doing so lets you have the best from both worlds:
  • You don’t need to write specific dashboard/GSS software for each project (and you get all the nice features that we described earlier).
  • And you don’t need to create & transmit a large JSON document from your microcontroller (heck, the person working on the microcontroller software doesn't event need to know what JSON is or how it works).
​
A JSON “map” document looks like this:
{
   "t":"%1",
   "g":[
      {
         "t":"Mission Status",
         "d":[
            {
               "t":"Runtime",
               "v":"%2",
               "u":"ms"
            },
            {
               "t":"Packet count",
               "v":"%3"
            },
            {
               "t":"Battery voltage",
               "v":"%4",
               "g":true,
               "u":"V",
               "w":"bar",
               "min":3.6,
               "max":4.3
            }
         ]
      },
      {
         "t":"Sensor Readings",
         "d":[
            {
               "t":"Temperature",
               "v":"%5",
               "g":true,
               "u":"°C",
               "w":"bar",
               "min":0,
               "max":80
            },
            {
               "t":"Altitude",
               "v":"%6",
               "u":"m",
               "w":"bar",
               "min":0,
               "max":3000
            },
            {
               "t":"Pressure",
               "v":"%7",
               "u":"KPa",
               "g":true,
               "w":"bar",
               "min":54,
               "max":102
            },
            {
               "t":"External Temperature",
               "v":"%8",
               "g":true,
               "u":"°C",
               "w":"bar",
               "min":0,
               "max":80
            },
            {
               "t":"Humidity",
               "v":"%9",
               "g":true,
               "u":"%",
               "w":"bar",
               "min":0,
               "max":100
            }
         ]
      },
      {
         "t":"GPS",
         "w":"map",
         "d":[
            {
               "t":"GPS Time",
               "v":"%10"
            },
            {
               "t":"Longitude",
               "v":"%11",
               "u":"°E",
               "w":"lon"
            },
            {
               "t":"Latitude",
               "v":"%12",
               "u":"°N",
               "w":"lat"
            },
            {
               "t":"Altitude",
               "v":"%13",
               "u":"m"
            },
            {
               "t":"No. Sats",
               "v":"%14"
            }
         ]
      },
      {
         "t":"Accelerometer",
         "w":"accelerometer",
         "d":[
            {
               "t":"X",
               "v":"%15",
               "u":"m/s^2",
               "g":true,
               "w":"x"
            },
            {
               "t":"Y",
               "v":"%16",
               "u":"m/s^2",
               "g":true,
               "w":"y"
            },
            {
               "t":"Z",
               "v":"%17",
               "u":"m/s^2",
               "g":true,
               "w":"z"
            }
         ]
      },
      {
         "t":"Gyroscope",
         "w":"gyro",
         "d":[
            {
               "t":"X",
               "v":"%18",
               "u":"°",
               "g":true,
               "w":"yaw"
            },
            {
               "t":"Y",
               "v":"%19",
               "u":"°",
               "g":true,
               "w":"roll"
            },
            {
               "t":"Z",
               "v":"%20",
               "u":"°",
               "g":true,
               "w":"pitch"
            }
         ]
      }
   ]
}

​As you can guess, Serial Studio will replace the %1,%2,%3,...,%20 values with the values at the corresponding index in a comma-separated data frame. The corresponding ​sprintf() format sent by the microcontroller for the given JSON map is:
/*KAANSATQRO,%s,%s,%s,%s,%s,%s,%,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s*/

You may have noticed some "w" keys in some places, these are used to build widgets (which are explained here). Finally, here is the obligatory GIF showing the usage of Serial Studio:
Picture
Pretty cool, right? If you are interested in using Serial Studio for your projects, here are the relevant links:
  • Github repo: https://github.com/Serial-Studio/Serial-Studio/
  • Documentation (GitHub wiki): https://github.com/Serial-Studio/Serial-Studio/wiki/Communication-Protocol​

The prebuilt binaries/installers for Windows, macOS & GNU/Linux are available through GitHub releases: https://github.com/Serial-Studio/Serial-Studio/releases/
​

Minimal example (with Arduino)

Suppose that we want to graph the an ADC reading with an Arduino & export the data to a CSV table. Here is the Arduino code:
#define ADC_PIN A0

void setup() {
   // Initialize Serial port at 9600 bauds
   Serial.begin(9600);
   
   // Configure analog input
   pinMode(ADC_PIN, INPUT);
}

void loop() {
   // Read voltage @ ADC_PIN
   int adc_value = analogRead(ADC_PIN);
   float voltage = adc_value * (5.0 / 1023.0);

   // Send current ms & reading through serial
   Serial.print("/*");        // Frame start sequence  [/*]
   Serial.print(millis());    // Add MCU runtime       [ms]
   Serial.print(",");         // Separator character   [,]
   Serial.print(voltage);     // Add voltage           [V]
   Serial.print("*/");        // Frame finish sequence [*/]
   
   // Wait 50 ms
   delay(50);
}

​Deploy this code to your Arduino and create a JSON file with the following contents:
{
   "t":"Minimal Example",
   "g":[
      {
         "t":"MCU Status",
         "d":[
            {
               "t":"Runtime",
               "v":"%1",
               "u":"ms"
            },
            {
               "t":"ADC reading",
               "v":"%2",
               "g":true,
               "u":"V",
               "w":"bar",
               "min":0,
               "max":5
            }
         ]
      }
   ]
} 

​Open Serial Studio & import the JSON file into Serial Studio by selecting the "manual" radio button in the top-left corner of the app & clicking on the "Change map file" button. Finally, select the appropriate COM port. If everything goes well, you should see a screen similar to this one:
Picture
If you click on the "Open current CSV" button, you will be able to see all the received information in an Excel/Calc table:
Picture
If you have any doubts, ideas or bug reports, feel free to add a comment, contact me or open up an issue at GitHub. Hopefully some random person on the internet will find this useful :)
Download Serial Studio (Windows, macOS & GNU/Linux)
19 Comments
adrian
1/13/2021 03:10:40 pm

I should probably use it before suggesting changes, but having been tempted to do something like this to enhance the display of arduino plotter output (but lacking the skill to do the PC part) I'd consider :

1. Adding a timestamp, alowing record / replay.
2. Allowing more than one tagged line from any given application, so channels can be sent at different rates
3. Or allowing partial lines of csv, with a numerical or alpha tag to indicate where in the expected stream the following values belong
4. Allow the json to be sent only occasionally on the stream, so it needs to be maintained in only one file yet not transmitted with every update

Reply
adrian
1/13/2021 03:22:08 pm

in fact, I think you already have (4) as the 'auto' mode - I had thought the data was embedded in the json

Reply
Alex Spataru
1/13/2021 08:40:16 pm

Hi, some of the features you mentioned are already supported (sort of):

1. The timestamp for each received packet is automatically added to the first column of the CSV. Also, we can 'replay' the contents of the CSV file by using the CSV Player function, which is available at the top right corner of the Serial Studio window.

4. I think that the terminology that I used is not very clear, "auto" mode expects the microcontroller to send the JSON for every packet. On the other hand, "manual" mode allows you to load the JSON from your computer, so that the Arduino/MCU only sends the comma-separated values (allowing Serial Studio to work with larger projects without overloading the resources of the Arduino/MCU).

As for the other features (2/3, & partially 4), I am already thinking of ways to do that. Serial Studio should have the capability to process different frame formats from the same serial device, and I am sure that the protocol has a lot of areas of improvement. I am just waiting for more feedback to know what things should be improved.

You are very welcome to check the progress, and contribute ideas & solutions at the Github repo (https://github.com/Serial-Studio/Serial-Studio).

Reply
Eduardo
1/18/2021 01:20:05 pm

As a suggestion, a clear button would be nice, because right now I'm using this solution only for monitor Serial response, because I think the serial monitor from arduino and esp32 a bit ugly.

Reply
Alex Spataru
1/20/2021 07:50:05 am

I've just added a clear console button with the latest commit. The feature will be available in the next release (v1.0.11), thanks for the suggestion!

Reply
Eduardo
1/25/2021 01:32:57 pm

Thanks for taking this suggestion. As I said before, this solution is really good looking and well designed and I hope that the community use it as primary Serial Monitor, because Arduino IDE and VS CODE serial monitor are not that appealing.

Could you implemment a toggle switch to toggle between to receive the control caracteres like CR or not? To me, this option can improve readability, because sometimes those chars are not important.

César
1/30/2021 09:21:49 am

Hola Alex: antes de nada enhorabuena por este maravilloso proyecto. Es genial. Mis futuros equipos que participen en Cansat España seguro que lo usarán y sacarán buen provecho de ello. La verdad es que es muy rápido y fácil generar la interface para el cansat.
Lo único que no me funciona es el widgets "map". Configuro el json tal como explicas en la wiki pero sale en blanco y sin datos. En cambio en el tablero si aparecen los datos de GPS. Uso la versión 1.011 sobre Windows.
Espero que me puedas ayudar a solucionar este inconveniente.
Y gracias de nuevo por este excelente trabajo

Reply
César
2/1/2021 02:46:27 am

Hi Alex: first of all, congratulations on this wonderful project. He is great. My future teams that participate in Cansat Spain will surely use it and take advantage of it. The truth is that it is very fast and easy to generate the interface for the cansat.
The only thing that doesn't work for me is the "map" widgets. I configure the json as you explain in the wiki but it comes out blank and without data. Instead on the dashboard if the GPS data appears. I use version 1.011 on Windows.
I hope you can help me solve this problem.
And thanks again for this excellent work.

Reply
Alex Spataru
2/1/2021 11:38:36 am

Hola César, muchas gracias por el agradecimiento. En cuanto al mapa, Serial Studio requiere la instalación de OpenSSL para mostrar los datos del mapa. El instalador de Windows debería de venir una distribución de las bibliotecas de OpenSSL. Sin embargo no he tenido chance de hacer pruebas extensivas debido a que en general uso macOS o Linux por cuestiones de trabajo.

Por el momento, usted puede descargar OpenSSL de https://slproweb.com/products/Win32OpenSSL.html, asegurase de descargar la versión de 64-bits.

Reply
Alex Spataru
2/2/2021 12:49:02 pm

Hola de nuevo, la última versión del software soluciona tus problemas con los mapas. El bug report de este problema se encuentra en: https://github.com/Serial-Studio/Serial-Studio/issues/19

César
2/2/2021 01:13:58 pm

Gracias de nuevo Alex. Quedó resuelto el problema con los mapas en la última versión (1.0.13). En los próximos días testearé el giroscopio y acelerómetro y si todo va bien, es la aplicación perfecta para los cansats.

Erkko
1/31/2021 11:52:05 pm

Is it possible to send data in fixed width "chunks" with each frame?

For example, suppose you're sending a bunch of 16 bit values really fast, say you're sampling an ADC at 20 kHz and you want to maximize throughput. You would want to drop the commas between the values and send out as many values at once as possible to minimize the overhead of the frame protocol, so what you want to do is tell the receiving end in advance that each frame contains n-number of two-byte values and append those to whatever data container they belong.

I was doing this by sending a simple stream of bits, but the problem is synchronizing the receiving end to the correct byte boundary. Eventually you get a skipped byte and then weird things start to happen.

Reply
Erkko
2/1/2021 12:23:24 am

Or, many times when you're dealing with IoT stuff, the amount of data you can send is rather limited, or you're trying to minimize transmission time to shut everything down quickly, so you don't want extra stuff in the frame. Every second the transmitter is on eats up your battery.

So if it's not in already, it would be useful to define a "raw" format that is then unpacked and broken down into bytes, or even into individual bits for that matter.

Reply
Erkko
2/1/2021 12:47:10 am

One use case for the fast data dump is when you have some sensor, like a microphone next to a running motor, and you want to build a system that takes 1 seconds of sound at a reasonable sample rate from the mic every few minutes and collects it for analysis - so you can listen if the bearings are going out or the fan is blocked etc.

So you're getting something like 10-20,000 data points, which won't fit in some simple and cheap MCU memory, so you have to stream them out as fast as they come.

Reply
adrian
2/1/2021 01:38:44 am

One way to do this is by escaping, or byte stuffing the stream.You put a framing character in, and if that's ever required as part of the stream data you 'escape' it by sending a 2-byte sequence of some other special character followed by a modified version of the framing character (eg the top bit flipped). Of course, you have to escape the escape character too.

This works well but it's important not to use a value that's commonly in the stream (like 00 or FF) or you can end up doubling the amount of data sent. If sending the data as binary you may also need to escape any characters special to the operating system, such as CR and LF.

Erkko
2/1/2021 06:11:38 am

Well, in my case it can be that the top byte can never reach 0xFF so there can never be 0xFFFF in the stream. This is because the ADC is actually 12 bits unsigned, so it will never produce that value. If it was signed int, then that would be -1 and that wouldn't work.

Steve Hageman
2/1/2021 08:09:19 am

Wow! You are my hero! :-)
Great job.....

Reply
Mark
2/2/2021 11:32:50 am

Really nicely done! The overall UI design is just beautiful and well thought out.

I do have connection problems, however. Specifically, I generally cannot get my microcontroller to connect simply using the Serial Studio UI. I see the correct COM port in the pulldown, but selecting it doesn't appear to do anything. However, if I use RealTerm to open the COM port, I see the serial stream from the device. I can then turn off the COM port in RealTerm and then successfully open the COM port in Serial Studio (yay!). Once the connection is successful, I can then use the COM port selector pulldown to toggle the stream on/off in Serial Studio and it works as expected. If the physically disconnect the microcontroller then reconnect, I then have to go back through the whole process again (using RealTerm to get an initial connection, etc).

Thank you for your hard work on this and opening it up to the public. It's something that I've been looking for for a long time! It is greatly appreciated.

-Mark

Reply
Yohan H
2/23/2021 11:33:04 pm

Hi Alex!
Thank you very much for sharing your amazing work!

I just started using Serial Studio, and I have few suggestions, especially regarding widgets :

(1) For the gps widget, could it be possible to display the track traveled by the gps point?

(2) I saw that the map could be tilted, as if it was in 3D, is it possible to also display the gps altitude in the map?

(3) Do you think that the accelerometer widget could be easily modified, to create a compass widget ?

Reply



Leave a Reply.

    Alex Spataru

    Website author

    Archives

    December 2020

    Categories

    All
    FOSS
    Microcontrollers
    Qt

    RSS Feed

Copyright © 2020 Alex Spataru.
All rights reserved.
  • Home
  • Blog
  • Projects
    • Serial Studio