For the last few weeks, I played around with controlling my LEDs over the network by mapping a MIDI keyboards dials to each of my color controllers settings.
Now I want to share my results and the code with you as well as some of the things I learned along the way.
Full HD version of the video above:
In this article, I will give an overview of the client software written in python, showcase a plug and play WiFi setup, how to announce and find the device in the network, and the basic principles of how to get an ESP32 to accept and act out some commands.
If you want to know more about the basic setup for the RGBW controller, check out this article.
And if you want to try this out yourself consider using this amazon affiliate link to get the same keyboard I have and help me pay my bills.
1. The Client
1.1 Reading data from the keyboard
To actually have something to send we need to have a client that reads the MIDI data, transforms it, and generates the command for the server.
We'll be reading the incoming data by parsing the incoming USB data and converting it into usable MIDI data using the midi subpackage of pygame.
I will be using pipenv for easier package management:
# Install pipenv
pip install --user --upgrade pipenv
# Install pygame
pipenv install pygame
# Enter virtualenv
pipenv shellNow we can have the pygame.midi package print us out all midi devices:
import pygame.midi
def get_devices():
devices = []
for n in range(pygame.midi.get_count()):
info = pygame.midi.get_device_info(n)
if (info[2]): # Check if the device has the input flag
print (n,info[1]) # (device id, device name)
devices.append(n)
return devices
pygame.midi.init()
print(get_devices())We will then connect to the MIDI device of our choosing by its device id. Listening in, we will ignore any key-presses and only work with the dials and button presses:
buttons = [*range(16, 23 + 1), 64]
dials = [*range(1, 8 + 1)]
def readInput(input_device):
while True:
if input_device.poll():
event = input_device.read(1)[0][0] # Strip away irrelevant data
if (event[0] != 176): # Check if this is a change event for a mode/control (this is device channel 1 only, you might want to check 176 to 191)
if (debug or event[0] not in [144,]):
print (event)
continue
if (event[1] in dials): # Check if it's a dial
dial_handler(event)
elif (event[1] in buttons): # Check if it's a button
button_handler(event)
else:
print (f"ID: {event[1]}, State: {round(event[2]/127,2)}")
send_data()
my_input = pygame.midi.Input(device_id)
try:
readInput(my_input)
except KeyboardInterrupt:
my_input.close()
Let's take a look at the handler for the dials:
dialdata = {}
def dial_handler(event):
pos = event[2] / 127 # Remapping to 0 to 1
print(f"Dial {event[1]} at position {round(pos, 10)}") # Mandatory print statement
dialdata[event[1]] = pos # Take a guess.. event[1] is the dial idSkipping the button code as it is only relevant for switching the color mode which is irrelevant for this article. Also removed some code that uses the second row of the dials for fine tuning purposes, for the same reason.
1.2 Compiling data to be send
The send_data function will now take the dialdata, transform it, and finally send it to the server:
stime = 0
last_fill = [-1, -1, -1, -1]
def send_data():
global stime # I know I know global vars are ugly but this is not clean code 101
global last_fill
ptime = int(round(time.time() * 1000)) - stime
if (cm and ptime > 10): # cm and ptime will be explained below
fill = [-1, -1, -1, -1] # -1 represents a "keep value" state
for i in range(1, 4 + 1): # Just filling the array with our dialdata
fdata = dialdata.get(i, -1)
if (fdata == -1):
continue
# I thought here was some fine tuning code somewhere...
fill[i - 1] = fdata
if (fill == last_fill): # Why would we send a new command if nothing changed?
return
last_fill = fill
f = fill
m = "manual" # For simplicity sake, check the source code for more interesting stuff
bridge.send_command(cm, "set", [m, *f]) # "set" and the m, *f array are just the type of the command and it's respective data.. more on that later
stime = int(round(time.time() * 1000))
First of all, what is this stime and what is it doing there?
Well, to not overwhelm the ESP by directly sending the status of the MIDI dials to it, causing it to queue up commands that will be overwritten by the newest one anyway,
we will have to limit the number of times we actually send the data.
In my case, I will wait 10 ms before sending the next command.
Which is coincidentally about the same time the ESP takes to update its color and become ready for the next command.
To be able to understand how we are sending data we need to understand what a command is first.
It is an identifier for the action that should be performed, followed by its respective options. It is then sent over a TCP connection to the server.
A command to turn on all LEDs looks like this:
set manual 1 1 1 1 \n
│ │ │ │ │ │
│ │ │ │ │ └> White value
│ │ │ │ └> Blue value
│ │ │ └> Green value
│ │ └> Red value
│ └> how should the parameters be interpreted?
└> set color to ...Note the newline character on the end as it will be essential for performance later on.
1.3 Sending the data
To construct the command from its parameters, encoding it to be sent as bytes and actually sending it we will use this code:
def send_command(socket, command, options):
socket.send(f"{command} {' '.join(str(x) for x in options)} \n".encode("utf-8"))But how do we get this socket?
This socket or cs from the send_data function further up stands for command socket and is basically our command line that gets injected into the send_command function to make it stateless and therefore easier to use.
To open this socket we first have to find out the IP of the ESP.
Thankfully the ESP is constantly screaming into the endless void of the network that it indeed exists. (more on that later) And if we are willing to listen to its desperate screams by opening a specific port that it broadcasts to we can get its IP:
from socket import *
def open_listener():
bclistener = socket(AF_INET, SOCK_DGRAM) # Some setup stuff
bclistener.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) # Some more setup stuff
bclistener.bind(('0.0.0.0', 24444)) # Listening on port 24444
return bclistener
def await_controller(bclistener):
while True:
message = bclistener.recvfrom(64) # Reading message from socket
if (message[0] == b'ThatLEDController online!'): # Checking for specific message
return message[1][0] # Returning ip
bc = open_listener()
ip = await_controller(bc)
Now that we got the IP we can plug that into a socket and open our command socket. Here is a handy function for that:
def open_sender(ip):
cmsender = socket(AF_INET, SOCK_STREAM) # Some setup
cmsender.connect((ip, 25555)) # Connecting to port 25555 at 'ip'
return cmsender
cs = open_sender(ip)Now that the client is implemented, let's build the server side, shall we?
2. The Server
2.1 Connecting to the network
The basic way to connect an ESP32 to the network would be to include the WIFI library, throw some (generally hardcoded) credentials against it and hope it connects:
#include <WiFi.h>
const char* ssid = "SSID";
const char* password = "Password";
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.println("Connecting to WiFi..");
}
Serial.println("Connected to the WiFi network");
}But luckily there is a library called WiFiManager that does this and much more for us with little to no configuration. It tries to connect to a predefined or saved network and when that fails, opens an AP on its own that that allows you to specify a network that it will then try to connect to:
#include <WiFiManager.h>
#include <WiFi.h>
#include <WebServer.h>
WiFiManager wifiManager;
void setup() {
// Configure wifi
WiFi.mode(WIFI_STA);
// Set dark mode
wifiManager.setClass("invert");
// Set manager to not blocking
wifiManager.setConfigPortalBlocking(false);
// Automatically connect to saved network
// or open network named "ThatLEDController"
// You can also pass this function a predefined network
wifiManager.autoConnect("ThatLEDController");
}
void loop() {
// Non blocking process function
wifiManager.process();
}OK, WiFi connection check, now how about allowing the client to actually find us?
2.2 Finding the ESP
To reliably connect to the device without relying on a hardcoded IP address, we will let the ESP tell it to us on its own. Well, not just us but rather the entire network.
We will use something called broadcasting for that, that allows you to send some data to a specific IP address in the network(most of the time something with 255 on the end) and it will be relayed to every client connected:
#include <AsyncUDP.h>
#include <WiFi.h>
// Non blocking wait helper
#define runEvery(t) for (static uint16_t _lasttime;\
(uint16_t)((uint16_t)millis() - _lasttime) >= (t);\
_lasttime += (t))
AsyncUDP udp;
void setup() {
// Setup wifi
}
void loop() {
runEvery(2000) { // Please don't spam the network
if (WiFi.status() == WL_CONNECTED) { // Check if we are connected
IPAddress broadcastip;
broadcastip = ~WiFi.subnetMask() | WiFi.gatewayIP(); // Get broadcast address for current address range
if(udp.connect(broadcastip, 24444)) { // Open connection to port 24444
udp.print("ThatLEDController online!"); // Send message
}
udp.close(); // Close connection again
}
}
}Moving on to the actual command server.
2.3 Setting up a basic server
Here is the basic setup for a simple single client TCP server running on port 25555:
#include <WiFi.h>
WiFiServer wifiServer(25555); // Open server on port 25555
WifiClient wifiClient;
void setup() {
// Setup wifi
wifiServer.begin(); // Start server
}
void CheckForConnection() {
if (wifiServer.hasClient()) { // Check if there is already a client
if (wifiClient.connected()) { // Check if the client is actually connected
wifiServer.available().stop(); // Reject new client
} else {
wifiClient = wifiServer.available(); // Accept new client
Serial.println("Client connected!");
wifiClient.write("Hey there!"); // Send welcome message
}
}
}
void loop() {
CheckForConnection();
}You can test that it works using netcat:
nc 0.0.0.0 25555Now there should be a server running that we can extend to accept some commands.
2.4 Receiving commands
First, we will have to read the command into a variable.
We will do this with a blocking function, as we have defined the commands to end with a newline which makes it faster than reading it in chunks. Add this part after the connection check:
if (wifiClient.available()) {
std::string cm(wifiClient.readStringUntil('\n').c_str());
Serial.println(cm.c_str());
}We will now check if the first three characters are actually "set" and if that's true pass is on to the handler function:
if (wifiClient.available()) {
std::string cm(wifiClient.readStringUntil('\n').c_str());
if (cm.rfind("set ", 0) == 0) {
setCmd(cm);
}
}The handler function will then split the command at the spaces and check if it is actually long enough. After that the respective values will be extracted and a new color is set:
std::string mode = "hsv";
std::array<int, 4> fill = {0, 0, 0, 0};
std::array<std::string, 4> modes = {"manual", "rgb", "hsv", "hsi"};
// Helper function to split string at delimiter
std::vector<std::string> split (const std::string& str, char delimiter) {
std::vector<std::string> tokens;
std::string token;
std::istringstream tokenStream(str);
while(std::getline(tokenStream, token, delimiter)) {
tokens.push_back(token);
}
return tokens;
}
void setCmd(std::string str) {
std::vector<std::string> tokens = split(str, ' '); // Split string at spaces
if (tokens.size() <= 1) { // Return if the command is to short
return;
}
// Get what could be the mode from the extracted tokens
std::string tmode = tokens[1];
// Loop over tokens and extract color, ignore if the data is negative
// resolution_factor is the pwm range
int tsize = tokens.size() - 2;
std::array<int, 4> color;
for (size_t i = 0; i < 4; i++) {
color[i] = i < tsize ? atof(tokens[i + 2].c_str()) * config::resolution_factor : fill[i];
color[i] = color[i] >= 0 ? color[i] : fill[i];
}
// Set mode to extracted mode, if extracted mode is valid
if (std::find(std::begin(modes), std::end(modes), tmode) != std::end(modes)) {
mode = tmode;
}
fill = color;
}And now there be light.
3. Conclusion and a look into the future
For a single device, this setup works great, even though it has some problems with reconnecting to the device. But that can easily be fixed by allowing new devices to override the already connected one.
I also added a preprocessor flag to my light controller that allows to deactivate anything but the command server, network code, and basic pin setup. This should save some resources when the direct interface is not needed.
In the long run, however, I want to expand the system to allow for more devices in the same network and create an interface to control them. (electron-based maybe?)
Furthermore, I might even try to connect the MIDI keyboard directly to the ESP as it technically acts as a USB host. But having to handle the USB to usable MIDI data conversion myself might be a bit overwhelming.
Anyway, that's all, thanks for reading!
