I have a not-so-secret technique that I use whenever I encounter a bug that evades my grasps. Or when I am lazy. Okay, mostly when I am lazy.
The idea is no more than applying the concept of binary search to source code:
Remove half the code
Compile and run
If the bug is still present, repeat with the remaining code
Otherwise, repeat with the code that was removed in 1
Of course, this naive approach is rarely practical. For once, removing half the code indiscriminately tend to make things not compile. Also, sometimes, a bug manifests because of the way the first half and the second one interact.
So, in practice, it looks more like:
Remove some big chunk of code
If that fails to compile/run, try another chunk, or reduce the size
Compile and run
If the bug is still present, repeat with the remaining code
Otherwise, put back the chunk of code, and try removing another one (can be a sub-chunk of the initial chunk)
Note: sometimes, you will still need to do mini-refactor to keep making progress
This works surprisingly well. And it is very satisfying to remorselessly hack through code that was written painstakingly over hours, months or even years. And then, getting to a small subset of code that clearly exemplify the bug.
Automate the Fun Parts
Software engineers might be the group of people who work the hardest to make themselves redundant.
I am guilty of spending way too much time automating tasks that only took a trivial amount of time
So, of course, someone already automated the process I described above. It is called C-Reduce. It is meant to find bugs in the compiler rather than in the compiled program, but the principle is the same. If your program is C and you can write a small script that test whether the bug is present, then it will automatically slash through your code.
To give you an idea, consider the code below.
#include <stdlib.h>
#include <stdio.h>
void f(void) {
int x = 42;
int y = 83;
int z = 128;
printf("%d\n", x + y);
}
void g(void) {
int x = 1;
int y = 2;
int z = 3;
printf("%d\n", x + y);
}
int h(void) {
return 42 * 43;
}
int main(void) {
h();
f();
}
#include <stdarg.h>
#include <time.h>
As well as the following Bash script.
#!/usr/bin/env bash
set -Eeuo pipefail
gcc -o main main.c
./main | grep -q ^125$
Then creduce test.sh main.c reduces the C source file to:
main() { printf("%d\n", 42 + 83); }
This was really a toy example, but that this program is very useful when you have large source files. For instance, C++ ones *shivers*.
Audio Millisecond Mistiming
I have been writing on a tool to practice Morse code using jscwlib, the library behind LCWO. I want to play an infinite flow of words in Morse code. The algorithm is the fruit of decades of research:
Play a word
When it has finished playing, play another word
While dogfooding, I realized that the speed sometimes increased unexpectedly. This was subtle enough that I did not notice the transition from normal speed to the higher speed.
Over my next practice sessions, I kept an eye ear on this phenomenon, and eventually realized that:
The space between the words did get shorter
The duration of the characters remained the same
From the two first points, I was able to devise a more reliable way to detect the presence of the bug. First, I made all the words just the letter “e”, which is a single “dit” in Morse. Second, I measured the time between the start of two words with new Date(). Finally, I showed the durations on the console.
This is not a very precise method but, it enabled me to see the duration of spaces reducing millisecond by millisecond over a few seconds. The effect became noticeable after a few hundred words were played.
With this, I could test the presence of the bug in mere seconds, instead of minutes of practice. So I went slashing.
After a short while, I found the culprit, wrote a fix, and opened a PR!
Conclusion
In the end, it took me much more time noticing the bug, understanding what it looked sounded like, and finding a way to check for it effectively, than actually identifying its origin in the source code.
The radio amateur service does not allow encryption of communications. This is a fundamental rule meant to ensure that the amateur service is not used for unintended purposes, such as commercial ones.
As a consequence, we cannot transmit passwords securely over the amateur frequencies. But we can use cryptographic authentication. Just not anything that would be used to hide the contents of the communications.
Cryptographic authentication is something that HTTPS/TLS allows out-of-the-box. We just need to make sure that we are not using encryption. In this article, we’ll see how to do this with Mozilla Firefox.
Forking Firefox
The code base of Firefox is an unwieldy Mercurial repository. Maintaining a fork would be cumbersome.
Git has won. And learning Mercurial for a small project adds quite a bit of friction, and I could not host on GitHub or GitLab. Of course, I can use git-remote-hg, but this adds another layer of abstraction, and another source of problems. I tried both, but it turned out it did not matter.
Since the official build process assumes that you are using the official repository with the official URLs and so forth, forking the repository itself would require also patching the build process. And I really did not want to touch that.
Another approach is to make a repository that just contains the patches to apply to the Firefox codebase, and a script that fetches the Firefox repository, applies the patches, and builds the project. This is gluon.
If you look into the HamFox repository, you will see that it contains just a few small files. The most important one is the null cipher patch itself.
The Patch
Grepping through the code for the TLS names of the cipher, we uncover an array sCipherPrefs that lists the supported ciphers. Here is an extract:
Each entry in the array is a CipherPref. The definition is given right above the array:
// Table of pref names and SSL cipher ID
typedef struct {
const char* pref;
int32_t id;
bool (*prefGetter)();
} CipherPref;
The first field is the string used to toggle the cipher in the about:config page. There is no point is creating a toggle for this cipher, so we’ll just leave it empty.
The second field is the identifier of the cipher; we find that the macros are defined in security/nss/lib/ssl/sslproto.h. When looking for the null ciphers, we find:
The SSL_* ones are just aliases for the TLS_* ones. We do want authentication. So, this eliminates TLS_NULL_WITH_NULL_NULL. We then have the choice between MD5, SHA-1 and SHA-256, and I opted for the last one, so TLS_RSA_WITH_NULL_SHA256.
The third one is a pointer to the function that tells us whether the toggle is enabled or not. There is no toggle. But we can just write a function that always returns true.
We replace the contents of the array with a single entry containing these fields, and we’re good!
To minimize the risk of conflict when the upstream code changes, I have put the other ciphers in a block comment instead of removing them altogether from the code. With this, we get:
diff --git a/security/manager/ssl/nsNSSComponent.cpp b/security/manager/ssl/nsNSSComponent.cpp
index 5844ffecfd..e2da79480a 100644
--- a/security/manager/ssl/nsNSSComponent.cpp
+++ b/security/manager/ssl/nsNSSComponent.cpp
@@ -1054,9 +1054,15 @@ typedef struct {
bool (*prefGetter)();
} CipherPref;
+bool AlwaysTrue(void) {
+ return true;
+}
+
// Update the switch statement in AccumulateCipherSuite in nsNSSCallbacks.cpp
// when you add/remove cipher suites here.
static const CipherPref sCipherPrefs[] = {
+ {"", TLS_RSA_WITH_NULL_SHA256, AlwaysTrue},
+ /*
{"security.ssl3.ecdhe_rsa_aes_128_gcm_sha256",
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
StaticPrefs::security_ssl3_ecdhe_rsa_aes_128_gcm_sha256},
@@ -1103,6 +1109,7 @@ static const CipherPref sCipherPrefs[] = {
StaticPrefs::security_ssl3_rsa_aes_128_sha},
{"security.ssl3.rsa_aes_256_sha", TLS_RSA_WITH_AES_256_CBC_SHA,
StaticPrefs::security_ssl3_rsa_aes_256_sha},
+ */
};
// These ciphersuites can only be enabled if deprecated versions of TLS are
As I have discussed previously, we can use cryptography over the amateur radio service, as long as we disable encryption. I have shown how to do this with SSH. Now, let’s see how to do this with HTTPS.
Modifying a Web browser, or even just a Web server, is an involved process, so we’ll start small, with the openssl command-line tool. On with crypto-crimes! No, not that crypto.
OpenSSL Server
The openssl command-line tool can be used like netcat to open a TLS session. To start a server, instead of:
$ nc -nlvp 4433
Listening on 0.0.0.0 4433
We’ll do:
$ openssl s_server Could not open file or uri for loading server certificate private key from server.pem 4067D713C67F0000:error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file 4067D713C67F0000:error:80000002:system library:file_open:No such file or directory:../providers/implementations/storemgmt/file_store.c:267:calling stat(server.pem)
Uh oh.
Right. We’re doing TLS, so we’ll need some cryptographic key.
If you are on a Debian-based Linux distribution (e.g. Ubuntu), you can always use the “snakeoil” key:
Note that we need to run the command as root to be able to read the private key. The -www option starts a basic Web server behind the TLS server, so you can test the setup by browsing to https://127.0.0.1:4433/. It will serve a Web page giving you the command-line you use as well as debugging information about the TLS connection.
Note: your Web browser does not know the emitter of the cryptographic key, so it will show a scary warning as shown below. This is expected, so you can just click on “Advanced…” and then on “Accept the Risk and Continue”.
Never trust the people at 127.0.0.1.
We’ll want to create our own cryptographic key instead of using the “snakeoil” one. Also, we’ll need a corresponding “.pem” file, or certificate, that will give a name (“CN” for “Common Name”) to this key. This is done with the following commands.
As with the “snakeoil” certificate, you can check the connection from your Web browser:
The warning sign on the padlock in the address bar is telling you that the connection might not be fully secure. Do not worry. It is going to become worse. Much worse.
OpenSSL Client
We’ve seen nc -nlvp 4433. Now, let’s see nc -v 127.0.0.1 4433:
openssl s_client
By default, it will try to connect to port 4433 on the local loopback. If you have a running s_server without the -www option in another terminal, you will be able to send messages from one another.
Note: if you do not remove the -www option from the s_server command, you will not see your inputs as you might expect, since they will be either ignored (server side) or sent as part of the HTTP protocol (client side).
Note: input is sent line-by-line, so you won’t see your message in the other terminal until you hit Return.
If you want to feel like a leet-h4x0r, you can use the s_client command to connect to any public TLS server. For instance, instead of clicking on the link https://perdu.com, you may run the following command
$ openssl s_client -connect perdu.com:443
Then, type the following and hit Return twice (you need to send a blank line).
GET / HTTP/1.1 Host: perdu.com
You should see the HTML for the Web page being printed in your terminal:
<html><head><title>Vous Etes Perdu ?</title></head><body><h1>Perdu sur l'Internet ?</h1><h2>Pas de panique, on va vous aider</h2><strong><pre> * <----- vous êtes ici</pre></strong></body></html>
Disabling Encryption
What we’ve done so far is basically the equivalent of the code shown below, except everything was encrypted.
To convince ourselves of it, we can use Wireshark. Once it’s started, select the any interface, type tcp.port==4433 as the filter, and hit Return.
Mind like a blank slate. Like a baby. Or me when I have to read reduced standard C++ headers.
Oh! And go play with the s_server and s_client command like we’ve seen above. You should see stuff like this:
Seriously, Wireshark is cool. It knows stuff.
This shows network packets being exchanged on the TCP port 4433 (the one used by default by s_server and s_client). The one I have selected corresponds to the transmission of “Hello World” from the client to the server. You can see it. It’s right here in the “Encrypted Application Data”. Okay, you cannot really see it, since it’s encrypted, by take my word for it.
Now that we have done all of that, we can ask ourselves the fun question: how can we make all of this pointless?
Let’s disable this all “confidentiality-preserving” technobabble “encryption” stuff.
First, let’s ask openssl to tell it what encryption modes it supports. On my computer, I get:
Each line is a possible setting (“cipher suite”). What matters is the Enc= part. All of these settings are using some mode of “AES” as encryption. “AES” stands for “Advanced Encryption Standard”. Booooring.
That’s more like it! We have several options to choose from. You see these Mac=MD5? I know we are trying to give a syncope to every security researcher on the planet, but we still respect ourselves. We’ll use NULL-SHA256. It’s the most secure of the totally insecure cipher suites. Trust me, I have a PhD in cryptography.
Let’s redo the s_server and s_client stuff again, with this cipher suite.
Note: we need to add the -tls1_2 option; otherwise, OpenSSL will offer TLS 1.3. The cipher suite we have selected is only available for TLS 1.2, and OpenSSL uses a separate option, -ciphersuites (yes, seriously) to configure TLS 1.3 cipher suites. Just add -tls1_2 and don’t think about it too much.
Hey, did you notice we were doing IPv6? That’s totally transparent, thanks to the magic of the OSI model. Wait no, TCP/IP is not actually OSI. Also, TCP and IP are not independent. Also learning the OSI model will probably never be useful to you. No, that had nothing to do with the contents of this article. Mostly.
Totally different, you see?
Actually, yes. 48656c6c6f20576f726c64 is the hexadecimal representation of the ASCII encoding of Hello World:
Who can’t read ASCII fluently?
Great! Now, we’re really back at Netcat-level security.
Note: sure, the client is checking the certificate of the server. And we could do all that with a certificate signed by Let’s Encrypt so that it’s not actually fully useless. Oh yeah, right, no, the Web browser is all panicky about the “no security” thing and won’t even let you enable the NULL-SHA256 cipher.
The error messages are getting worse. It means we are going in the right direction.
Client Authentication
Okay, okay, wait up. There is one more thing we can do with this, and it might serve some kind of purpose. The browser is checking the authenticity of the server. But what if, like, the server checked the authenticity of the client, man?
Yes, we can do this!
Modern Web browsers are a bit fussy when we disable encryption, so the command-lines in this section will not do so. That way, you can try in your Web browser. But, know that you can combine with the NULL-cipher, and test with the s_client command.
First, let’s tell the server to check the authenticity of the client. We tell it to use server.pem as the authority for client certificates. That is, to be considered valid, client certificates must have been signed by the private key of the server.
On the server’s side, you should see something like:
$ openssl s_server -key server.key -cert server.pem -verify 1 -CAfile server.pem
verify depth is 1
Using default temp DH parameters
ACCEPT
depth=1 CN = localhost
verify return:1
depth=0 CN = John Doe
verify return:1
…
subject=CN = John Doe
issuer=CN = localhost
Note: alternatively, you can add the -www option to s_server and navigate to https://127.0.0.1:4433/. At the bottom, you’ll see no client certificate available. You can change that by loading the client certificate in your Web browser.
Note: to add the certificate in Firefox, go to:
“Settings”
“Privacy & Security”
“Certificates”
“View Certificates”
Your Certificates”
“Import…”
Select the client.p12 file
Give an empty password
Okay, but… Why?
Good question! When we do this, we are allowing the client to authenticate using a secret file on the computer (the private key). This could be used in an actual Web service to let people connect to their account without using a password. That’s a good thing. Since we’re going to disable encryption, we better not have passwords flying on the air!
Conclusion
Now, all of this was a bit academic. But it has allowed us to test out the waters of disabling encryption and using client authentication. It’s also going to be useful when we want to check that a modified Web browser can connect to an s_server instance with NULL encryption, or that we can authenticate to an HTTPS service with a client certificate.
In a previous article, I explained that encryption is not allowed on the radio amateur service, but that we could still use public-key cryptography. In particular, we could use SSH for authentication, but without encryption. This would allow secure remote control of base stations, satellites and more.
In this article, I will discuss how we can do that in practice without reinventing the wheel. The result is the HamSSH project, which forks OpenSSH to provide an encryption-free, but still authentication-secure implementation of SSH.
Building OpenSSH
The most common implementation of the SSH protocol is OpenSSH. To build the project from the sources, we run the commands below:
$ sudo apt install autoconf gcc git libssl-dev make zlib1g-dev
$ git clone https://github.com/openssh/openssh-portable
$ cd openssh-portable
$ autoreconf
$ ./configure
$ make -j$(nproc)
Once this is done, we can generate a server key with ssh-keygen, start the server and try connecting with the client. You should be able to do that by running the commands below in two terminals:
You may try using -o "Ciphers none" on the server’s command-line, and -c none on the client’s, but it will fail because the null-cipher is not enabled. Our first task is to find how to make it available.
Thanks to the -d option on the server’s command-line, we can check what encryption cipher have been used:
From this, we can take a guess and search for the string chacha20 to locate the place where the ciphers are chosen. We quickly find an array named ciphers, in cipher.c. It looks promising:
We can see that the none cipher is actually listed. Now, we need to find out how to allow it. The sshcipher struct that make the elements of the array is defined right before:
The name field is self-explanatory. The fields block_size, key_len, iv_len and auth_len are parameters for the cipher. They are not used by the null-cipher, so they are set to zero in ciphers. The last field, evptype or ignored looks optional, so it probably does not matter for what we want to do.
That leaves flags. Given the definition of CFLAG_INTERNAL and the associated comment, it seems reasonable to think that this macro controls what ciphers should not be made available in normal operation.
If we look further, we find two functions that use this macro:
The code is pretty straightforward. The first function iterates over the list of ciphers, and filters out those whose flags field matches the CFLAG_INTERNAL macro. The second one takes a comma-separated list of cipher names, and check that they all correspond to known ciphers in the list, and that they are not internal ciphers.
If we change CFLAG_INTERNAL to 0, that should enable none as a normal cipher.
Note: you might want to set CFLAG_INTERNAL to ~CFLAG_NONE to disable the real ciphers, but this just makes sshd segfault. This is because the default list of ciphers becomes invalid. We’ll get back tot his.
After rebuilding, restarting the SSH server, and connecting with the client, we get:
It’s actually normal: we enabled the null-cipher, but it’s not necessarily selected. In fact, it is still not listed as one of the options by either the server of the client. We can force this by adding -o "Ciphers none" to the server’s command-line, and -o none to the client’s. Then, we get:
But we want this to be the default behavior. In fact, we would like to ensure we do not use encryption by mistake, so we want to disable the other encryption modes.
Disabling Encryption
Let’s look into how the option Ciphers is handled
If we look in the other source files containing the string chacha20, we find this in myproposal.h:
“KEX” stands for “KEy eXchange”. So KEY_…_ENCRYPT relates to the stage where the server and the client decide how to encrypt the communications. The macro is used in the servconf.c and readconf.c source files, where the server and the client (respectively) detect the default values for the Ciphers option. Let’s try changing the string to just "none", and starting the server and the client without any particular option.
Let’s make sure this does not happen. We can now come back to the definition of CFLAG_INTERNAL and set it to ~CFLAG_NONE. Since the default cipher list is "none", this will not make the server crash. And if we try to use anything but null-cipher, we get:
// server
Bad SSH2 cipher spec 'chacha20-poly1305@openssh.com'.
// client
Unknown cipher type 'chacha20-poly1305@openssh.com'
Done!
Niceties
We now have a null-cipher-only implementation of an SSH server and client. Let’s see what we can do further to avoid mistakes.
Changing the Names
First, let’s make sure we do not confuse the regular SSH binary, and our null-cipher only one. We’ll also want to avoid overwriting the regular ones by mistakes, and we might want to use different configurations files.
HamSSH prepends a “ham” suffix to all the binary and configurations files. So the user configurations files can be found in ~/.hamssh, and the global configurations files are in /usr/local/etc/ham*.
Changing the Default Port
Regular SSH uses the port 22 by default. We will want to use another one for our null-cipher-only SSH. Since 21 is already taken by FTP, I opted for 23. It is normally used for Telnet. Since Telnet is even more obsolete than FTP, that should not be a problem. And users can always use another port by using the regular SSH options.
I also like the idea that it reminds us that all communications are in the clear, just like with Telnet.
Disabling Password Authentication
The SSH protocol is still useful without encryption because it gives us public-key encryption. But we do not want to risk the user mistakenly typing a password when trying to log in.
In other words, we want to force the PasswordAuthentication and KbdInteractiveAuthentication options to No. The simplest way is to overwrite options.password_authenticationoptions.kbd_interactive_authentication after the command-line arguments and configuration files have been parsed, in sshd.c (server) and ssh.c (client):
/* Force public key authentication */
options.password_authentication = 0;
options.kbd_interactive_authentication = 0;
Conclusion
With just a few tweaks, we have a robust and versatile tool for remote control compatible with amateur radio regulations. We have forcibly disabled encryption on both the server and the client to avoid using encryption by mistake, disabled password authentication to avoid spilling secrets carelessly, and changed names and the default port for compatibility with regular SSH. All we need to do now is to use it!
The C programming language is a gift that keeps giving. Today, we are going to see how a seemingly banal and common operation can hide unfathomable depths of unmentionable horrors.
Woes with Integer Coercion
What is the problem of the code below?
// returns the closest integer to f
int float2int(float f) {
return (int) f;
}
It’s written in C.
A Rust enthusiast
That’s… a good point.
But let’s assume we actually want to write C, for some forsaken reason. And we do not care whether we convert up, down or sideways.
There is still a fundamental issue with this code. What happens when we call float2int with INFINITY?
Yup, that’s undefined behavior. Converting a non-finite value (i.e. an infinite or a NaN) to an integer is undefined behavior.
But that’s not all! What should the largest finite floating-point value (3.4028234664e+38) convert to? It’s much bigger than INT_MAX, the value that int can represent (it does not matter whether you are on a 32, 64 or 128 bit architecture, really).
Maybe we could just say that all the floating point-number larger than INT_MAX should be converted to INT_MAX when converted? But alas, it makes for unwelcome surprises down the line, when you realize that the rounded value is not always within 1 of the original value.
Thankfully, the C standard always has the solution whenever there is a hint of ambiguity:
When a value of integer type is converted to a real floating type, if the value being converted can be represented exactly in the new type, it is unchanged. If the value being converted is in the range of values that can be represented but cannot be represented exactly, the result is either the nearest higher or nearest lower representable value, chosen in an implementation-defined manner.
If the value being converted is outside the range of values that can be represented, the behavior is undefined. Results of some implicit conversions may be represented in greater range and precision than that required by the new type (see 6.3.1.8 and 6.8.6.4).
In other words, we’re just going to say all the values that are not quite that convenient trigger undefined behavior. The “range of values that can be represented” by int is the interval [INT_MIN, INT_MAX]. Attempting to convert any float value from outside this range to the int type is undefined behavior.
We’ll just have to check for that!
Someone has not learned their lesson
Woes with Range Checking
We are just getting to the actually tricky part. Let’s have a look at a seemingly robust implementation:
#include <limits.h>
#include <math.h>
int float2int(float f) {
if (isnan(f)) {
return 0;
}
if (f < INT_MIN) { // also filters out -INFINITY
return INT_MIN;
}
if (f > INT_MAX) { // also filters out +INFINITY
return INT_MAX;
}
return (int) f;
}
For the sake of simplicity, we are just providing arbitrary values for the values that we cannot convert. Other implementations might choose to return a status indicating whether the conversion is possible or not.
But, of course, there is a bug lurking in there. And it is extremely easy to miss.
And, this time, it is not caused by an undefined behavior, but by another surprising “feature” of C: implicit type conversions.
To understand what I mean by that, we need to look at what the conditions in the code above really mean. The first one is fine, let’s look at the other two: f < INT_MIN and f > INT_MAX. In both, the left operand is a float, and the right operand is an int. Processors rarely have such built-in operations. So a conversion must be taking place. The way it happens in described in the “usual arithmetic conversion”:
[…]
if the corresponding real type of either operand is float, the other operand is converted, without change of type domain, to a type whose corresponding real type is float.
Thankfully, we only need the part I have quoted here. In short, both operands get converted to float. Let’s look at our conditions again.
if (f < (float) INT_MIN) { // also filters -INFINITY
return INT_MIN;
}
if (f > (float) INT_MAX) { // also filters +INFINITY
return INT_MAX;
}
Let’s look at the first condition. What is the value of (float) INT_MIN? Let’s assume 32-bit 2-complement int type. Then INT_MIN is -2³¹. A float can represent this value exactly. You can check it out with an online converter. So (float) INT_MIN is -2³¹ as well. No problem here, this conversion does not change the value, and this code does exactly what we want.
With the same assumption, INT_MAX is 2³¹ – 1. And there comes the catch: a float cannot represent 2³¹ – 1 exactly. If you put “2147483647” in the online converter, you will see that it is rounded to a float whose value is actually “2147483648”. It makes sense: a float trades off being able to represent all integers in [-2³¹, 2³¹ – 1] in order to cover a much wider range and many more magnitudes. In other words, the actual value of (float) INT_MAX is INT_MAX + 1. The condition is actually doing:
if (f > INT_MAX + 1) { // also filters +INFINITY
return INT_MAX;
}
Note: if this were C, this code would contain an undefined behavior because of the signed integer overflow that happens when evaluating INT_MAX + 1. This is just pseudocode for illustration purposes.
Since the upper bound is offset by one, the code fails to filter out the half-open interval (INT_MAX, INT_MAX + 1]. If the input parameter f were to take its value from this range, converting it to int would be undefined behavior. Now, is there any float value in this interval? Yes! INT_MAX + 1 of course (it’s the only one).
Conclusion
The “robust” implementation does not handle 2³¹ properly. This is not just a theoretical oddity. I encountered this issue in critical code in production while working at TrustInSoft. Such bugs will easily stay under the radar and avoid even the most intensive test campaigns. Only formal methods help in detecting and avoiding such errors.
The worse thing is that, since this is undefined behavior, you might encounter it many times, and never now, corrupting data in unknown ways. Things that could have been a runtime error become the entry door for silent bugs and security issues.
Yes, this is the 2005 picture from Wikipedia. No, things have not changed that much since. Physics still work the same.
Amateur radio operators, or “hams” for short, benefit from special privileges to this end. The radio spectrum is a crowded place; yet hams are allowed to transmit at various frequencies spread between 135,7 kHz and 250 GHz (not all bands are visible in the chart). Radio equipment is normally strictly regulated to limit interference and operations in unauthorized bands; yet hams are allowed to modify transmitters as they please, or even build and operate their own from scratch.
The allocation of the radio spectrum in the United States. The allocation is similar in other countries. You might notice there is not much free room. Click to enlarge.
It would be very tempting for a commercial operator, for instance, to make use of these privileges for their own benefit. To avoid this, there is one very important restriction: all communications must be done in the clear.
Code is Law
The exact wording from the international regulations is:
Transmissions between amateur stations of different countries shall not be encoded for the purpose of obscuring their meaning
Article 25, paragraph 2A from the Radio Regulations edited by the ITU (emphasis mine) (yes, the website is very slow, which is a bit sad for the International Telecommunication Union)
Note: you might think that “of different countries” means that hams can encrypt their communications within a country, and that can actually be the case. However, the article is meant to bind countries to a common standard, not to restrict what happens within each country. In practice, each country will decide whether they want to allow it locally. And, in general, they don’t.
Encoding is the mapping of information to a particular representation. For instance, Morse Code and ASCII are encodings that map the letters of the alphabet to short and long signals, and to numbers, respectively. AM and FM are encodings that map analog signals to other analog signals, that is to say modulation.
Encoding is necessary for any form of communication, so it is of course allowed in general. What is forbidden is to represent information in a way that hides its meaning.
The ASCII encoding maps the uppercase and lowercase Latin letters, as well as the Arabic numerals, some punctuation marks, and control characters
Note that the wording does not say “cryptography”. This is intentional. Cryptography is just a set of techniques which can be used to hide information. It does not have to be this way, and this is not the only way to do so. This is about intent.
If you are doing Morse Code, SSB, FT8, or experimenting with your own super-advanced error-correction encoding that is meant to be usable by anyone, you would be in the spirit of the law. On the opposite, if you are hiding information in the low bits of a digital image, or using a custom protocol only you and your buddies know about, or using a proprietary protocol whose specification is not public, you are going against the spirit of the law.
Steganography is the idea of hiding information in plain sight
Note that the difference between “experimenting with a new protocol” and “designing a protocol restricted to just a few people” is slim. What matters is intent. A way to show good faith would be to include a link (using plain voice or Morse code) to an open source repository that anyone can easily install to decode the rest of the communication. To be clear, that’s not an original take.
What about cryptography? It should obviously be banned, right? The entire point of this discipline is to hide information.
My plant in the audience to ask a convenient question at a convenient time for my transition
Mostly, yes. But we are here to tinker, so let us see what this “mostly” entails.
Confidentiality and Authentication
The two main use cases of cryptography are confidentiality and authentication. Mostly everything else derives from these.
They are mostly used together. For instance, the most widespread implementation of cryptography is indubitably TLS, the protocol used to secure all communications of the Web. This the S in HTTPS. This is also what enabled online shopping, and grew the Web from a niche protocol to visit personal homepages to the societal backbone it is today.
The green padlock is telling you that the communications with the website are made secure using cryptography. It does not tell you that the website itself is trustworthy, however.
Nowadays, when you connect to a website, your Web browser will ensure two things.
Note that authentication is necessary if you want confidentiality. Without at least some basic form of authentication (e.g. sharing a private key), encryption will only give confidentiality against passive attack models.
However, authentication on its own could definitely make sense. And the purpose of authentication is not to obscure the meaning. In particular, a cryptographic signature is a piece of data that anyone can decode and interpret. The information it contains is simply the fact that the associated message was indeed sent by its author, not by someone else.
Note: the usual rebuttal is this would allow an attacker to send the same message again. However, this is a well-known topic and easily solvable.
To be clear, I am not the first person to think about doing this. Nor the second:
Why not? The purpose of amateur radio is to experiment with communication technology. Knowledge can be an end in itself. But still, having some ideas of what practical applications could look like can be motivating, so I list some here.
Remote Control. Since 2004, Radio Regulations include an exception to article 25.2A for the control of satellites. Indeed, going to orbit is hard¹, so you want to be able to service satellites remotely. But if anyone can remotely turn off your satellite, or make it dive into the atmosphere, you’ve got a problem. So you need some way to authenticate the commands. But you do not actually need encryption. This means that remote control over amateur radio is possible for things other than satellites. Remote base stations, or repeaters, for instance.
Authenticated QSOs. Such schemes could be used to authenticate contacts, or QSOs, to prevent people from spoofing call signs. This could serve a similar purpose as PGP or GPG for email.
Contesting. Possibly the most popular activity among hams is contesting. That is, contacting as many other operators in the world in a given time frame. Nowadays, scoring is done by having all the operators send the log of their contacts to the contest’s website. If K1AA says they have contacted F6ZZ at 28,500 MHz around 14:00 UTC, and F6ZZ says they have contacted K1AA at the same frequency at around the same time, we can probably assume that this actually happened. Unless K1AA and F6ZZ are colluding, of course, but this is hard to prevent. But if, for some reason, F6ZZ is unable, or forgets, to upload their logs, then the contest will assume that K1AA made a mistake, and subtract points. With authenticated QSOs, K1AA could provide strong evidence that the contacts happened as described. Contests already routinely require the exchange of serial numbers, so it might not be such a stretch to implement. Instead of an incrementing number, each operator might have a computer generate a short code for each contact.
Wow. Much radio. So frequency. Many QSOs.
Online services. Now that digital modes and packet communications are finally being used in amateur radio, it is possible to use protocols such as HTTP. But, without authentication, the use cases are as limited as the Web in the early 1990s. With authentication, we could envision message boards and editable wikis with secure user accounts.
These are just the few examples I could think of, but there might be other relevant applications. Of course, there are wrinkles to be ironed out, but I do not pretend to have it all figured out.
Null Cipher
How do we use that in practice? We could roll our own crypto, or not. On the one hand, it is deceptivelyeasytomakemistakeswhenimplementingcryptographybyoneself. On the other hand, the purpose of amateur radio is to experiment. So we might play around with custom cryptographic protocols over the air, but maybe we should use proven ones for actual persistent use. Let’s see what we could use for this.
TLS. The first obvious candidate is the omnipresent TLS. It is mostly used to authenticate servers, but you can also use it to authenticate clients. This has already been discussed in the past in the context of amateur radio to authenticate clients for remote base stations. Using TLS with the “NULL” cipher is acommontopic, and it is not very hard to tell OpenSSL to do so (-cipher NULL-SHA256:@SECLEVEL=0). Modifying some web server to serve content with the NULL cipher is more involved, but probably not that hard.
However, both Mozilla Firefox and Google Chrome have removed the NULL cipher from the list of supported ciphers. In other words, they will refuse to connect without encryption. Fixing this would involve modifying their source code (easy), navigating their complicated build process (annoying), distributing them to the users (hard), and maintaining the fork (who wants to do that anyway?). Unfortunately, this is probably necessary, if we also want to prevent users from inadvertently using encryption by connecting to HTTPS websites. I’ll probably look into it at some point, but this is a significant project.
Me writing some random text through an SSH connection
Wireshark intercepting the raw communications for the SSH sessions over the network. As you can see, the text is visible in clear. The [0m, [1m, [27m and so on are ANSI escape sequences. The rest are the SSH handshake and the cryptographic signature, which are easier to read when displayed as such.
Conclusion
To sum up, I like the idea of using cryptographic tools to provide proper authentication for remote control and enable new use cases through amateur radio. Of course, there is significant work to be done before we can get to there, but it’s nice to know that we do not need to start from scratch.
The one thing that C cares about is speed. Seriously, you might not get the right results, but at least, you will get them fast.
Let’s look at two common ways to represent a vector in C.
struct vector {
unsigned long size;
int data[42];
};
The first one is to just use the built-in array type. This approach is simple, but implies choosing the size of the vector when compiling. The program cannot decide the size depending on user-input or some other runtime parameter.
Note:Variable-Length Arrays (VLA) do allow declaring arrays whose length is decided at run-time. However, this is limited to local variables and is an optional feature since C11.
struct vector {
unsigned long size;
int* data;
};
The alternative is to not use an array at all. Instead, we use a pointer to a memory buffer that will contain the data. This memory location can be created with malloc, for instance.
Pointers can be used like arrays almost transparently, which brings (the “almost” means that most C programmer are paranoid about using sizeof on expressions). The example below shows how accessing an element at some index uses the same syntax in both cases.
But although the syntax is the same, the meaning is different. Look at the assembly code generated to implement vector_index. Indexing the built-in array uses a single MOV instruction; indexing through a pointer uses two MOV instructions.
It makes sense. rdi contains the address to the struct and rsi the value of index. In the first case, we shift by 8 bytes to skip the field size, then we move by index * 4 bytes to select the right array element, then we read the value at this memory location (indicated by the bracket in the Intel assembly syntax). In the second case, we need to first recover the value of the field data, at an offset of 8 bytes from the beginning of the struct, then read at the right offset from the address it contains.
In other words, using a pointer instead of a built-in array is twice as slow! Okay, not really. Any modern CPU is going to recompile, pipeline, prefetch and out-of-order execute this such that it really won’t matter. But it used to! And it might still matter on microcontrollers.
In fact! I did some benchmarks, and, although there was no difference on my laptop (CPU model i5-4210H), I did see a tremendously large difference on my Arduino Leonardo (microcontroller model ATmega32u4). Indeed, the code with the additional indirection through a pointer was 10 % slower.
…
Yeah, I did expect it to be worse than that. But still, 10 % was not something to scoff at in the previous millennium. Also, CPUs at the time might have seen a larger difference than my ATmega32U4 released not that long ago.
In summary, embedding a built-in C array is fast, but inconvenient. Using dynamic allocation with malloc is more flexible, but not as fast. How can we make our compiler generate faster code while keeping the convenience?
Well, we can lie to it.
Lies. So Many Lies.
Let’s look at how we will typically create a vector with the first approach:
struct vector {
unsigned long size;
int data[42];
};
struct vector *vector_new(unsigned long size) {
assert(size == 42);
struct vector *ret = malloc(sizeof(struct vector));
ret->size = size;
return ret;
}
Notice how malloc is just another C function, that can take any value as an argument at runtime? Notice how we are forcing ourselves to give it a compile-time value (the size of our struct)? What if we actually allocated the size we wanted?
For instance, if we allocated sizeof(struct vector) + 100 bytes, that would mean we would still have the 42 elements from the definition of struct vector, but we would have 100 additional bytes, enough to store 25 more 4-byte integers!
Since we are going to lie to the compiler about the size of the actual array anyway, we make the base size as small as possible. Since the C standard does not allow 0-length arrays (“An array type describes a contiguously allocated nonempty set of objects”), we just use 1. Then, when we create an instance of our vector, we are going to make sure to actually have enough memory for size elements. Since there is already 1 built-in, we need to allocate for a size - 1 elements in addition to the base struct.
C developers embraced this hack, wrote many fast programs, and lived happily ever after.
Well, kind of. I mean, if you write this, your program is probably going to be fast. Just as fast as a fixed-size array. But maybe it won’t. And maybe it will allow random people to take control of your computer. Or maybe it will blow up the spaceship. Who knows?
Yes, we did allocate enough memory to do what we want. But this is not the point. We lied to the compiler. And compilers are very delicate, gullible beings. It will believe you. It will believe that your array is actually of size 1. And it will do unmentionable things to your code, because this assumption is false.
Can you see the difference? We just removed the 0.
That’s all?
Someone who wasted too much time reading this; but worry not, it took even longer to write!
Yup. This syntax tells the compiler to not assume anything about the size of the array. You can still use it as usual. But ensuring that you allocate enough memory, and that you do not go out-of-bounds, is up-to-you.
So, you’ve been programming for one month, or for ten years, and you have gotten familiar with Murphy’s and Sturgeon’s laws. And you are wondering if there is some way to catch all the bugs, not just spray-and-pray with regression/unit/integration/system/acceptance testing.
Or you are a curious onlooker, wondering why all these programmers cannot make a piece of software that works reliably, what with apps doing unexpected things and with all the security breaches you hear off.
First, what are we talking about? Essentially, a bug is anytime a system does not do what it is expected to:
Today, I would like to focus on the last one. It looks like it should be the more tractable, since the scope is well constrained. But it is still a non-trivial problem.
What we want is two-fold:
Specify what we expect the program to do
Verify that the program meets our specification
Even the first step is tricky. For the sake of simplicity, we are going to reduce the scope of what we want to achieve, and focus on a well-delimited set of things our program should do: have a well-defined behavior.
What does this mean? Let us say you wrote a program that does some kind of scientific computation, expect you used meters instead of kilometers. Then, the result is going to be wrong, of course, but in a predictable way: the result will be off by a factor 1,000. If you run the program again the same way, you should get the same result.
On the contrary, if your program contains undefined behavior, it could do unexpected things. It might output the right value; it might output a wrong value; it might crash; it might allow an attacker to take control of your computer. And it might behave differently every time you run it. Worse: it might run perfectly fine during all your tests, but output the wrong value when it is being used to pilot a real plane with real passengers.
How can we know if a program contains undefined behavior? First, we need to talk about programming languages.
Why is C an “Unsafe” Language?
When people talk about “undefined behavior”, they are usually referring to undefined behaviors in the C programming language. There is a reason why this particular language is criticized as being “memory-unsafe”, and why so many security vulnerabilities stem from code written in C. And that reason is that the C language leaves the door open for many bugs.
To understand why, we must first ask ourselves what “the C programming language” refers to. Who decides what it is, and what if someone does things differently?
Historically, C was loosely defined and had many dialects. Every compiler author would do things a bit differently. The people writing a compiler to compile “C” code to 8080 assembly made different choices than the ones targeting PDP-11. The same program could end up doing different things on different processors. Things were messy.
So, the C compiler writers, and the C developers, and the processors manufacturers joined together and agreed on a common definition for what “C” really meant. Thus was born the C standard.
The C standard had to try to align things between people doing very different things. And these people cared a lot about performance. Processors at the time were about a hundred times slower than the ones today. And I am only talking about clock frequency here; nowadays, each CPU core can do much more in a cycle than the CPUs at that time, and there are now many cores in a modern CPU. It makes sense that they really wanted to keep compiling C code to efficient assembly, to have programs run at decent speeds.
Now, what happens if the code tries to access memory it is not supposed to? Well, the compiler cannot just add runtime checks everywhere, since that would have a negative impact on performance. And it cannot always know in advance that this will happen, since it would require it to solve arbitrary complex mathematical problems.
Because of this, the C standard did not require either. Instead, if the code does such an invalid operation, it is said to have “undefined behavior”.
So what? If code crashes, I’ll notice the problem. If it outputs the wrong result, my tests will notice it.
Simplicius, my imaginary straw-man C developer
Wait a minute, renegade figment of my imagination! Undefined behavior means that the compiler is allowed to produce code that does anything. In particular, it is allowed to assume that undefined behavior never occurs in the problem, and deduce things from that. And you can prove many things if you assume something which is false.
Let’s take an example. The code below uses a toy authorization function that just checks whether the user id (uid) . As you might have already guessed from the topic of this article, there is an undefined behavior in this code. First question: can you see it?
#include <stdio.h>
int authorized_uids[] = {0, 1001, 1005, 1012};
int is_authorized(int uid) {
for (int i = 0; i <= 4; i++) {
if (uid == authorized_uids[i]) {
return 1;
}
}
return 0;
}
int main(void) {
if (is_authorized(4242)) {
puts("You are now logged in as root.");
} else {
puts("You are a lowly common user.");
}
}
Spoiler: answer below.
The loop is written as for (int i = 0; i <= 4; i++), so i takes the values 0, 1, 2, 3 and 4. Then we use it as an index to read from an array: authorized_uids. However, this array only contains 4 elements, at indices 0, 1, 2 and 3. There is no index 4 in this array. Trying to access index 4 is an out-of-bounds access, which is a type of undefined behavior.
Second question: what will happen when we compile and run this code? You can actually do it, but do try to have a guess first.
The universal answer is: anything can happen. Since there is an undefined behavior, the program could do anything. However, there are two common things compilers will do in this case.
Option 1. It can translate this C code to assembly in a rather naive way. For an x86 processor, it would translate authorized_uids[i] as an assembly instruction reading memory at the address authorized_uids + i.
This might seem a bit obvious, but the compiler does not have to do that. Even when there is no undefined behavior, the compiler is allowed to emit any code it wants, as long as it produces the same result. For instance, the compiler might find a faster series of instructions that achieves the same results. This is what “optimizing” mean.
If we then run the compiled program, the CPU will try to read at authorized_uids + i. Most likely, however, this will still be in the valid memory of the program. Indeed, memory is usually allocated generously (segments for the stack, pages for the heap). So, the assembly instruction will not cause a crash; it will return some value. Most likely, the value will not be 4242, so the test will fail, and the user will not be authorized.
This is the middle column in the compiler explorer below. You can see at the top that the source code for is_authorized contains various assembly instructions. And, at the bottom, you can see the output of the program: “You are just a lowly common user.”.
Option 2. Now, let’s have a look at the rightmost column. The output is “You are now logged in as root.”! This is not what this program was supposed to do at all! And this is not just a fluke. It happens every time you run it!
Then, why is the result different this time? If you look closely, you will notice that the only difference is that we passed the option -O2 to the compiler. It tells the compiler to try harder to optimize the program.
In fact, if you look at the generated assembly, you will see that the compiler optimized the function to something extremely efficient:
is_authorized(int):
movl $1, %eax
ret
This means that the function is_authorized just always returns 1. Wat?
Let’s try to understand the reasoning of the compiler. It sees:
int is_authorized(int uid) {
for (int i = 0; i <= 4; i++) {
if (uid == authorized_uids[i]) {
return 1;
}
}
return 0;
}
This is equivalent to:
int is_authorized(int uid) {
if (uid == authorized_uids[0]) { return 1; }
if (uid == authorized_uids[1]) { return 1; }
if (uid == authorized_uids[2]) { return 1; }
if (uid == authorized_uids[3]) { return 1; }
if (uid == authorized_uids[4]) { return 1; }
return 0;
}
But we can assume that there is no undefined behavior in the code. So uid == authorized_uids[4] must never be evaluated. So the compiler can safely assume that the function always exits at one of the first 4 return statements. In other words, it always returns 1.
But couldn’t the compiler at least warn about this?
Someone rightfully annoyed with the state of the universe
Fighting Undefined Behavior
If you run gcc with -Wall -Wextra -Wpedantic or even clang with -Weverthing, you still get no warnings about the undefined behavior!
Why don’t they warn about this? Because this is harder than it looks. The compiler cannot always know where an UB will happen. Let’s have a look at the example below.
// hour ∈ [0, 23]
// minute ∈ [0, 59]
// second ∈ [0, 59]
int time_to_seconds(int hour, int minute, int second) {
return hour * 3600 + minute * 60 + second;
}
If a compiler had to ensure the absence of undefined behavior when compiling the above code, they would have to assume that hour, minute and second contain any possible value from INT_MIN to INT_MAX. The compiler does not understand comments written for humans, and it does not understand variable or function names, even when they are chosen appropriately.
To make the code safe, the compiler would have to introduce runtime checks in the compiled code to make sure that no overflows occur during the computation. In practice, that would imply 2 comparisons and 2 branches before each of the 2 multiplications and each of the 2 additions. But C was not designed for safety. It was designed for speed. All these comparisons and branches add up. On a simple benchmark, I observed that a safe version would be between 2 and 3 times as slow as the unsafe one.
For the same reason, if the compiler had to warn for every potential undefined behavior, developers would get flooded with false positives. In the example above, the compiler would have to emit a warning for each arithmetic operation. With many false alarms, the developers would have to get in the habit of just ignoring all the warnings.
But I do not care about performance that much! I want runtime checks everywhere. Or at least, I want to see the warnings! I set -Wall -Wextra -Wpedantic -Wconversion -Weverything. I really do want to know when there are potential undefined behaviors in my code.
Let’s address each of these.
Runtime checks. The clang compiler does have the ability to add runtime checks! ASan (Address Sanitizer) looks closely at the way memory is allocated, deallocated and used. UBSan (Undefined Behavior Sanitizer) looks at a few additional things, such as invalid arithmetic operations. Both of them work by adding instructions to the resulting binary executable, which will check everything. Since the program has more things to do, it runs slower. Valgrind/memcheck works directly on the compiled program, and inserts its own checks when running it. It also makes running the program much slower. These tools are able to catch a few types of issues every time they actually happen in the program. However, it is often non-trivial to understand why it happened. Using the function from the previous example, compiling with clang -fsanitize=undefined time-to-seconds.c and running the program would result in something like:
time-to-seconds.c:5:17: runtime error: signed integer overflow: 2147483647 * 3600 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior time-to-seconds.c:5:17 in
time-to-seconds.c:5:33: runtime error: signed integer overflow: 2147483647 * 60 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior time-to-seconds.c:5:33 in
Syntactic checks. Although they do not understand what your program does, compilers are able to notice suspicious patterns in the source code. For instance, they can see if (x = 42) and emit a warning, since this is a common mistake where the developer should have written if (x == 42). Modern compilers now include an extensive set of such patterns to warn the user about potential mistakes. Some tools are specialized in finding such patterns in the code, such as clang-tidy or cppcheck. Since they do not change the way the code is compiled, they incur no slow down when actually running the program. However, they can only find common mistakes, or they would risk emitting too many false warnings. In the example, no linter emits any warnings, since it would be likely to be a false positive. However, they do warn of more obvious cases:
warning: integer overflow in expression of type ‘int’ results in ‘1’ [-Woverflow]
39 | INT_MAX * INT_MAX;
| ^
If we want to go further, we need to look at formal methods. The idea is to use mathematical and systematical approaches to find all the undefined behaviors in a provable way. This means that such methods even allow you to prove the absence of undefined behavior (when there is none, of course).
I want to talk about two techniques from formal methods in particular. I am familiar with them because the company where I work, TrustInSoft, sells an industrial tool to apply them to real code used in critical applications.
Abstract interpretation. The idea for this one is intuitive. Essentially, we are going to interpret the source code in an abstract machine. This abstract machine will interpret each statement from the code exactly as specified by the standard. If undefined behavior can happen, a warning can be emitted. Essentially, this can be thought as an implementation of the abstract machine as described by the C standard.
The interesting thing is that you can go even further by having the abstract machine work with sets of values instead of singular values. For instance, you can analyze a function for all possible inputs (the values for each parameter being represented as a set of actual values), instead of just one at a time. For instance, let us consider the example below.
int abs(int x) {
if (x < 0) {
return -x;
} else {
return x;
}
}
This command interprets the source code in an abstract machine, starting from the abs function. By default, it assumes the argument can be any possible value (any integer from INT_MIN to INT_MAX). When checking the program for all these possible values, it notices a problem in one specific case: when x is INT_MIN, computing -x is undefined behavior, because it would be bigger than INT_MAX. Indeed, usually INT_MIN is 2³² and INT_MAX is 2³²-1, because of the way integers are represented. If you are interested in knowing how these programs manage to work with huge sets of values, look up interval arithmetic.
Weakest Precondition. The other technique is much more general and can actually be used to verify arbitrary facts on programs, not just to find undefined behaviors. For instance, with this, you can prove that your implementation of a sorting algorithm actually does what it is supposed to do. However, it also takes much longer to learn to use it, and to analyze a real program. If you are actually interested in it, I can only recommend the excellent tutorial written by Allan Blanchard.
What About Other Languages?
We explained why C is an unsafe language. But are the other languages unsafe?
C++. This is essentially a superset of C. And it adds its own undefined behaviors. It does introduce features to avoid doing mistakes common in C code, but the complexity it brings also hands another footgun to the user.
Java, Python, Ruby. These languages work by being compiled to byte code, which is something similar to assembly, except that it is not run directly by the processor. Instead, it is run by the corresponding virtual machine. In other, each language targets a single architecture. While C needs to encompass CPUs whose memory model vary significantly, these languages can specify strictly what should happen. They do not care as much about performance, either. The result is that, if you try to access out of the bounds of an array, you do not get undefined behavior. You get a nice, clean, exception telling you what the problem is, and a call stack telling you where it happened.
Rust. Now, Rust is an interesting case. Like C, it is designed to be fast. Unlike C, it is designed to be safe. It achieves this thanks to the concept of unsafety. By default, checks are added wherever they are needed, and dangerous operations are simply prohibited. You can turn off these checks to do lower level operations and improve performance where needed, but you have to do explicitly. The point is that unsafety is opt-in, so developers know to be careful with undefined behaviors.
As you can see, we can do quite a bit about undefined behaviors. There are very fast, very simple-to-use tools, that might find common mistakes, and there are very advanced, and exhaustive tools, that require more involvement. And there are languages trying to make safety the default.
There is still a long way to go until software becomes as reliable as it could be, but we’re getting there, one byte at a time.
We run tests. Unit tests. Integration tests. System tests. Acceptance tests. And where possible, we write software to do these tests for us. Usually, that means unit tests and integration tests. Usually, such tests won’t guarantee that our code is without defect. But at least they give us confidence that it does work for the cases we do tests. If we change something and the tests still pass, we probably have not broken something major. This let us iterate quickly without having to double and triple check whether a change broke one of many features. But even then, starting the tests and waiting for the results is burdensome, so we might get complacent and forget to run them before releasing our changes.
So we get even more software to remember that for us. That’s what we refer to as “continuous integration”, or CI for short. Whenever we submit a new feature, a bugfix, or even if we just want to slightly change the shadow under a button, the CI will run the tests and make sure we have not broken anything. For instance, in the CI of Kepler Project, I run linters to find obvious defects in code, static analyzers to find less obvious defects, as well as a suite of unit tests. I plan to add integration tests at some point, but it is slightly annoying to run them in graphical software.
Continuous Delivery
Once we are confident our changes will not accidentally launch the nukes, we want to actually release them to our users. But software developers are lazy, and they will spend hours automating a task that only takes minutes. In the end, this is the next step after continuous integration, so we call it “continuous delivery” (CD).
An added bonus of a well-designed CD is that you will be able to tell exactly what code was used to create a specific release. For instance, in Kepler Project, the CD starts whenever I create a new git tag. The release it generates uses the tag as the version number. Since a git tag points to a specific state of the code, I can easily check out the code corresponding to a version, to locate a bug, for instance. I can also browse all the changes between two versions.
CI/CD
If you have any kind of non-trivial project, you will want to at least run a CI. And even a basic CD will probably help you in tracking changes from a released piece of software back to the corresponding code in your code history.
199820082018 2022 will be the year of the Linux desktop. Definitely sometimes before 2028. Maybe before 2038? Ok, maybe not. In any case, I am the most comfortable working (and studying, and procrastinating, and gaming, and procrastinating) on Linux (yeah, yeah, GNU/Linux or GNU+Linux, I know). Actually I am a weakling running Debian with the non-tiling WM Xfce4 (I could as well install Xubuntu at this point).
Anyway, the point is that I still need to target Windows as a platform for the 96 % of users who have not yet seen the light. But I do not really want to have to reboot to switch to a different operating system and maintain multiple developing environment. Besides, I need to build my project in the CI/CD pipeline for automatic testing and release building, but I really do not want to deal with a Windows virtual machine in the cloud.
If only I could build for Windows without running Windows…
Cross-Compiling
Well, it’s a thing. Actually, compiling for X without running on X is a thing. It’s even pretty much mandatory when you cannot run a compiler on the target system. So, if you are developing for Arduino, you are most likely compiling from amd64 (Intel and AMD microprocessors) towards avr (Atmel’s microcontrollers). It’s called cross-compiling.
What makes cross-compiling not completely trivial is that the file hierarchy and dependency management are mostly built around the assumption that compiled code will be run on the current system. So, on my computer, binary libraries in /usr/lib are amd64, /usr/bin/gcc compiles to amd64 and apt installs amd64 binaries by default. Debian does support various architectures, so, I can tell apt to install arm64 libraries, and it will install them in /usr/lib/aarch64-linux-gnu/. Binaries of different architectures, however, will conflict: both amd64 and arm64 Firefox will want to be /usr/bin/firefox. But, in any case, I do not want a gcc compiled for arm64! I want a gcc compiled for amd64 that will compile to arm64. For this, I have to install gcc-aarch64-linux-gnu. With this, I can easily set up a working environment where I will be compiling for arm64 from my amd64 computer.
Cross-Compiling to Windows
But wait, I do not want to cross-compile to arm64! I actually want to cross-compile from amd64 to amd64, but for Windows! Does that even make sense? It does! If I install mingw-w64, I get a new gcc at /usr/bin/x86_64-w64-mingw32-gcc (actually a symbolic link to /usr/bin/x86_64-w64-mingw32-gcc-win32). And then:
$ cat hw.c
#include <stdio.h>
int main(void) {
puts("Hello World!");
}
$ x86_64-w64-mingw32-gcc hw.c
$ file a.exe
a.exe: PE32+ executable (console) x86-64, for MS Windows
I actually do get a PE file, an executable for Windows!
Good, I can compile code for Windows. Now I need to install my project’s dependencies, such as GLFW3. But there is no such thing as libglfw3-mingw-w64-dev. In fact, there are only a few libraries available for this platform in the Debian repositories:
libassuan-mingw-w64-dev
libgcrypt-mingw-w64-dev
libgpg-error-mingw-w64-dev
libksba-mingw-w64-dev
libz-mingw-w64-dev
libnpth-mingw-w64-dev
For everything else, you’ll need to compile the dependencies yourself and install them in /usr/local/x86_64-w64-mingw32/. Yup, no magic wand here. And of course, you will need to cross-compile them to Windows. And fix the bugs you will encounter while doing so.
Pro-tip: write yourself a script to build and install these dependencies. It will serve as documentation for when someone (probably future you) wants to install the development environment. And you will have to do it one way or another if you want to cross-compile in Docker for your CI/CD pipeline.
Running Windows binaries from Linux
How do we run the “Hello World!” example from earlier? Easy!
$ wine a.exe
[useless warnings]
Hello World!
Yup, that’s it.
Okay, okay, it is still useful to test the Windows build on an actual Windows or, at the very least a Windows VM since it sometimes behave differently.
Anyway, that’s all for today. And don’t think too much about floating point determinism when cross-compiling.
