Categories
Morse Code

Learning Morse with Koch

If you take an interest in learning Morse code, you will quickly hear about “Koch method” and “Farnsworth method”. You will also read many opinions stated with absolute certainty, often contradicting each other. Some advice is straight out non-actionable.

I have recently read Koch Dissertation on Learning Morse Code. Yeah, I have unusual ways to spend my time. Anyway, the reason for this is that I wanted to actually understand what Koch actually claimed, and what he actually proved.

I start with a rant about reading a badly scanned German paper from almost 90 years ago, and a small digression about who Koch is, but you can directly skip to the main part of the article. Or directly to the conclusion.

The Rant

There is an English translation of the dissertation, but it is a machine-generated one. Although significant effort has been done to make it look nice, the content is often too mangled to be usable.

The only version I could find of Koch’s dissertation is an old scan. Although it was uploaded in 2020, the file was created in 2008. Unfortunately, this is a time when JBIG2 was a thing1. Sure enough, when extracting the images from the PDF, I see various files ending in “.jb2e” or “.jb2g”.

Internet Archive serves a text version of the document. Since the document was typewritten and not handwritten, the OCR actually did a pretty good job. However, since it was clearly OCR-ed after the conversion to JBIG2 took place, some characters are mixed up, in particular:

  • h and b
  • n and u
  • c and e

Of course, most of the time, the resulting word did not make sense, and I just needed to fix the typo. In German. By the way, do you know about agglutination? Oh, and since it’s 1935, it uses various words that are obsolete today. Also, it’s orthography reform of 1996, so “ß” is still widely used. The OCR did not like that.

Thankfully, it’s 2023, and machine translation has removed all the borders, and made using foreign languages transparent. Or not.

Maybe it’s German, but neither DeepL nor Google Translate could make natural sentences out of the transcript. At least, they were mostly grammatically correct. But, between the fact that they still followed German word order and expressions, and the fact that they poorly cut sentences (sometimes duplicating them), it was far from a piece of cake.

Of course, after the transcription and the translation, we are not done. The argument itself is not a shining piece of literature. It was written by an engineer for a limited audience. On a deadline. And a typewriter.

Koch himself often repeats himself (in separate paragraphs, so it’s definitely not the machine translation), uses interminable sentences (the translation did not change the length), sometimes keeps the actual point implicit, and sometimes omits information altogether.

Extracting the gist of this paper was like peeling onions: each layer you remove only makes you cry harder.

Dr.-Ing. Ludwig Koch

Finding information about Ludwig Koch is not easy. It does not help that there were many other people named Ludwig Koch, including several doctors, and even a Prof. Dr. Ludwig Koch, born in 1850. And another teaching today. One thing to keep in mind is that our Koch was not a Doctor of Philosophy (PhD). His dissertation granted him the title of Doctor of Engineering, which would make him Dr.-Ing. Ludwig Koch.

What we do have to work with is the curriculum vitae attached to the end of the dissertation, where Koch states that he was born in 1901 in Kassel.

There was a “Dr-Ing. Koch” in the Nazi’s club2 of the technical university of Dresden in 1944, but it looks like it was Paul-August Koch, who studied textile and paper technology. The only other reference I have been able to find online of Morse-Koch is a 1971 document referring to “Dr.-Ing. Koch Ludwig”, which would match his title. He would have been 69 at the time.

Koch defended his thesis at the technical university Carolo-Wilhelmina. The main reporter was Prof. Dr. phil. Bernhard Herwig, who studied psychology3 and was mostly interested in studying working conditions for the good of the fatherland4. The co-reporter was Prof. Dr.-Ing. Leo Pungs, who was interested in more technical aspects of radio engineering as the dean of the Department of Electrical Engineering5. For the record, it looks like Prof. Dr. Phil. Bernhard Herwig was in the Nazi’s club, but Prof. Dr.-Ing. Leo Pungs was not6.

I tried to clarify whether Koch was actually interested in Morse code in and by itself, or was only using it as an arbitrary example to apply Arbeitspsychologie. He did have a degree in electrical engineering, but I found no evidence that he continued studying the learning of Morse code after he defended his thesis. In fact, there is no other trace of him in the academic record.

We think it was the same Koch famous for nature recordings

HackADay

No he’s not. Sound-recordist-Koch was born in 1881.

Koch was able to teach a class of students to copy code at 12 words per minute in under 14 hours. However, the method wasn’t often used until recently.

HackADay

I do not know why they think that “the method wasn’t often used until recently”. Koch’s method was known in the United States at least since The Learning of Radiotelegraphic Code in 1943.

Koch Dissertation

The bold parts are a summary of the summary I posted before. The rest are my comments.

For other summaries of the dissertation, look here, or here. I did my own summary independently of these, and before reading them.

We study how holism can help improve work conditions.

First, we need to tackle the fact that the dissertation is heavily oriented in favor of Gestalt psychology, or holism, which was very in at the time. Koch worked under the supervision of Prof. Dr. phil. Bernhard Herwig, who was mostly concerned with improving work productivity for the fatherland (see previous section), but it does not look like he was particularly focused on holism. This point would require more investigation, but my understanding is that Koch was somewhat biased towards holism due to the historical context, but not overtly.

When experienced radio operator send Morse code at a speed lower than 10wpm, the resulting timings are distorted

However, this was done with only 4 subjects. They do show the same overall pattern, but there are significant differences between them. So, it is hard to conclude with anything as precise as a “10 wpm threshold”. The associated graphs would tend to show that the distortion is still very visible even at 16 wpm (80 chars/min). The main point stands, but the point where sending is slow is very debatable.

Experienced radio operators have difficulty receiving Morse code at lower speed

This is associated with the graph below. Again, the main point is pretty clear. The accuracy of reception falls off a cliff as the speed decreases. However, the speed where this happen is again debatable. 90 % is already a pretty accuracy for an experienced operator. That’s one error every tenth character!

Success rate of experienced operators receiving low-speed standard Morse code. The abscissa is the speed in chars/min and the ordinate is success rate in percent.

If we look more closely at the graph, we notice that the curves bend downwards somewhere above 10 wpm. And if we pay attention, we notice that there are actually very few data points from which the curves are drawn. There is not enough information to know when the accuracy falls below 95 %, for example. There might be multiple data points in the upper right corner, but it’s hard to tell without the raw data, which is not available.

In short, slow speed is terrible, but there is no clear evidence that 10 wpm and above should be considered good enough.

  • Naive: first, learn all the characters at a slow speed, then increase the speed
  • Farnsworth7: first learn all the characters at a high speed, but with ample spacing, then decrease the spacing, then increase the speed
  • Koch: first learn the rhythm at 12 wpm, then learn 2 characters, then add other characters one at a time and finally increase the speed

Because this matches my intuition, I would like to say that the previous experiences show that the naive approach is clearly mistaken. But, looking at it objectively, it’s not what the experiences show. What they do show is that using low-speed Morse is not strictly easier than using high-speed Morse, to the extent that experienced operators do not automatically master low-speed Morse.

The main point of Koch can be summed up in two statements:

  1. At lower speeds, we need to think, while higher speeds rely on automatic recognition
  2. Starting to practice Morse with the thinking-mode is a waste of time

Regarding statement 1, it is hard to find explicit evidence, but it is at least sound, from the fact that operators just do not have the time to think about each individual dit and dah at high speed. However, this does not preclude the possibility of progressing to automatic recognition at lower speed. Also, at higher-speed, it is definitely possible to think about what you hear, just on a more selective basis.

For statement 2, I want to highlight the fact that the thinking- and automatic- modes directly map to the cognitive and autonomous stages in Fitts and Posner’s model. Thus, there is nothing inherently unnatural in progressing from thinking-mode to automatic-mode. The harsh plateau when switch from the former to the latter might just be a requisite passage in the learning process8.

In this part, the arguments of Koch for statement 2 are not supported (nor invalidated) by evidence. The evidence comes later in the paper from comparing Koch method with the other procedures.

Harder characters (X, Y, P, Q) should not be learned at the very first. Koch uses L, F, C, K, R, P, X, Y, Q, D, B, G, A, N, Z, U, V, W, J, H, S, I, E, O, M, T.

That makes sense, at least to avoid frustration. As far as I know, Koch did not systematically investigate the learning order, so that’s a minor point.

Same for similar characters (E/S/H/5, U/V, D/B, G/6, W/J)

Same as above.

Training sessions should be spread out in time instead of packed together, and should not exceed half an hour.

Spoiler: this is the truest claim of the paper. But it is not surprising. The spacing effect has been known from even before Koch was born. In fact, Koch cites a paper from 1897 on this exact topic.

The experience was conducted with students, tradesmen and businesswomen with two or three sessions a week, so it was at a disadvantage relatively to military training

As one knows, psychology is the study of psychology students. To be fair, Koch states that it was not just students. But, since we do not have the raw data, we cannot tell whether it was 10 % students, or 90 % students.

In any case, this argument is kind of weird. Koch argues that his study was biased in a way that should have made good results harder with relation to military training. Morse code training is offered to anyone who joined the military, while student are a select class of individuals, especially at the time.

In addition, he just argued that short sessions spread out in time were more effective than long, packed sessions (like the military does).

Two-tone method: using different tones for the dits and dahs helps at the start of the training

This is an idea that I see very rarely when I read about the Koch method online. Actually, I think the only instance is at LICW. I find it very appealing, since it sounds like it could really help with recognizing the characters more easily at first, and thus improve the feedback loop for learning.

However, the evidence for its effectiveness provided in Koch dissertation is very weak.

Learning curve for the single-pitch (dashed line) and two-pitch (solid line) methods. The abscissa shows the number of half-hour sessions, and the ordinate the number of characters mastered.

The argument is that the two-tone curve is always above the one-tone curve. However, there are a number of issues:

  1. We do not know how many participants there were in each case, so we cannot assess the significance of the result
  2. The curve have lots of variations, which suggest random differences were not smoothed out by the law of averages
  3. The relation between the two curves do not show a continuous advantage, since the spread does not increase over time. In fact, they even merge for a time!
  4. Even if we ignore the fact that they merge, the spread is mostly set during the second half-hour. We expect to see the most variance at the beginning of the training, when students are not familiarized with the process, and differ in their learning patterns. So it might explain the spread entirely.

Again, the two-tone method could actually be useful, but we cannot tell from the evidence. If anything, we do learn that the advantage is marginal at best.

In fact, LICW did try the two-tone method, but they found no advantage:

we experimented with introducing two-tone characters to new students and the feedback was overwhelmingly negative

Long Island CW club

The characters were added as H, F, A, B, G, C, D, E, CH, L, K, I, O, T, M, P, Q, R, S, N, U, W, V, Y, X, Z (no J)

With the new methods, students take 14 hours to master Morse code, instead of 10 weeks

This is definitely the most important claim of the paper. Military training typically uses several hours every day for months on end. Reducing this to only 14 hours, every if spread over several weeks, is phenomenal.

Well, 14 hours with 3 half-hour sessions a week is still about 10 weeks. So the whole gain would be about spreading the sessions instead of packing them. But still, let’s say that the claim is that this is only made possible thanks to the new method.

This is supported by the two following graphs9.

Learning curve for the Farnsworth method. The abscissa is the number of weeks of practice; the ordinate is the speed attained by the student.
Learning curve for the single-pitch (dashed line) and two-pitch (solid line) methods. The abscissa shows the number of half-hour sessions, and the ordinate the number of characters mastered.

Alas, there are a number of confounding factors that are not addressed.

One. In the second graph, we can see that the training completed once students had mastered the 26 letters of the alphabet. This excludes digits, punctuations, which are necessary for normal operation. But the first graph is actually not based on Koch experiments. It was “kindly provided by the Deutsche Verkehrsfliegerschule”, a military-training institution. Of course, we have no more information, so we do not know how many characters they had to learn.

In comparison, it took me less than 20 hours to reach 26 mastered characters (lesson 25 at LCWO) at 20 wpm, but 35 more to master all LCWO lessons. I did follow the Koch method, in the fact that I trained at 20 wpm from the get-go, without extra spacing. But the point is that, at 26 characters, I was still very far from being done.

Of course, the first graph could be about learning only the 26 letters, but it’s never stated so, and we have no way to check.

Two. The target speed is set to 12 wpm, which is actually pretty low. Koch himself states that the passing speed is usually at 20 wpm:

Radio communication in the ship service is under the same conditions, i.e. perfect mastery of Morse code at 100 char/min.. The examination conditions of the radio operator course are, for example:

“Keying a telegram of 100 words of 5 letters each in 5 minutes on the Morse apparatus in perfect form. Receiving a code telegram of 100 words and an open text of 125 words of 5 letters each in 6 minutes; transcribing the latter on the typewriter.”

Koch claims that going from 12 wpm to 20 wpm would be relatively easy, since all the characters have been learned in the automatic-mode. However, this was never tested, so there is no concrete evidence showing this. As an illustration, I took 55 hours to master 20 wpm (with 41 characters), but I am barely reaching 28 wpm 30 hours later.

Three. Remember how, in the previous quote, it said “perfect form”? Koch sets the bar at “less than 10% error rate”. It took me 44 hours to reach lesson 40 of LCWO at 10 % error rate, but still 11 more hours to get it down under 5 %.

And now, for the most damning issue.

Aptitude test: those who fail to master 4 characters at a speed of 60 char/min in the first 2 half-hours never mastered all characters.

This could indeed be a useful aptitude test. However, it points to a fundamental flaw with the study. What does it mean by “never”?

Does Koch mean that these students were allowed to continue the practice for months until he wrote his dissertation? And, more importantly, that he took their learning time in the above graph?

Or does it mean that they were allowed to continue the practice until the 28th session? In that case, it should show as a value strictly lower than 26 for the number of mastered letter at the end of the experience.

Or does it mean that they were removed from the experience early, when they gave up, or when it became clear that they could not keep up?

In any case, this is never addressed in the paper. Yes, military trainings eliminate students aggressively as they fall behind. And it could even be that Koch had a much higher success rate. But we do not know. At all.

Conclusion

Koch brings interesting ideas. However, none are supported by the evidence he brings forth. Except one, which is to use spaced repetition. This one is actually well-known and supported by evidence. Koch himself recognizes this.

To stress the point: it is very well possible that some parts of this method actually have a positive impact, but we do not have enough evidence from this paper to conclude. I have read many times people say that using Koch method, or Farnsworth method, or whatever method helped them. But, until we have concrete evidence, we cannot conclude that one approach is better than the others.

Then, how should I learn Morse code?

Just use Koch method on LCWO at 20 wpm. Lower if it is too fast. Practice 10 minutes a day. Every day. And keep at it.

But you just said that Koch method was not supported by evidence!

Yup. But it is not disproven either, and no other method is proven to work better10. Don’t get stuck in analysis paralysis trying to find the one best method. Use Koch because it’s the one for which there are the most tools, including LCWO, the most convenient one. Or just because I said so. Use 20 wpm because that’s the default in various places. Or because that’s what I did. But don’t worry about lowering it if you really cannot manage.

What does matter is actually practicing. So spaced repetition and discipline.

How long does it take to learn Morse Code?

Count 50 to 100 hours if you want to be comfortable at 20 wpm. Count a minimum of 10 to 20 hours for the bare basics if you’re aiming for 12 wpm.

This is a personal question but on average students get on the air doing their first QSO in 3-4 months.

We do not have requirements but recommend as a minimum 2 classes per week and 15–20 minutes daily practice.

FAQ of Long Island CW Club

We teach the code at 12 words per minute. This speed was determined by Koch to be optimum for initial learning of CW.  It is also optimum for making first contacts on the air.

Description of Morse code training at Long Island CW Club

Should I learn the Visual Representation?

Sure, if you find it fun. It’s a pretty easy goal, and that could be quite enough if you want to shine in escape games. If you intend to actually use it over the air, don’t wait until you have learned it, and just start practicing what you actually want to do.

Should I Learn Mnemonics?

No. Seriously, they’re all terrible, and it’s not that hard to learn the 26 letters, even if it’s just for an escape game.

  1. JBIG2 is a “clever” image compression format in that it notices when two parts of the image looks similar, and only store one. So, if two characters look a bit similar in a document scan, it could decide to merge them, making them identical. This obviously did not go well. ↩︎
  2. Dresden 1944–1945 program, page 31 of the document, page 33 of the document (note: “NSD” stands for “Nationalsozialisticher Deutscher Dozentenbund”) ↩︎
  3. Carolo-Wilhelmina 1929–1930 program, page 13 of the document, page 12 of the PDF, left half ↩︎
  4. Braunschweig during the National Socialism, pages 118–134 ↩︎
  5. Carolo-Wilhelmina 1929–1930 program, page 8 of the document, page 10 of the PDF, left half ↩︎
  6. School year 1944–1945 program, pages 21, 22 of the document, page 15 of the PDF, right half and page 16, left half ↩︎
  7. It was only named “Farnsworth method” much later though, as analyzed by the Long Island CW Club ↩︎
  8. I do not claim this is the case. But, so far in the paper, we have no evidence either way. ↩︎
  9. Koch also refers to another study, Methoden der Wirtschaftspsychologie (Handbuch der biologischen Arbeitsmethoden, page 333) which have a graph similar to the first one. However, the plateau is described has happening at 14 wpm, not 12 wpm. It also goes up to 22 wpm for reception, not merely 16 wpm. ↩︎
  10. Again, based on Koch’s dissertation. Other research might actually prove that one method works better. I have just not read one yet. ↩︎
Categories
Morse Code

Koch’s Dissertation on Learning Morse Code

This post is a summary of the dissertation that Dr.-Ing. Ludwig Koch submitted in 1935 to become doctor engineer. This is what is referred to by “Koch method” in the context of learning Morse code. I will publish my commentary in another post.

You can find the scan, the transcription, and the full translation in the dedicated repository. The scan comes from the Internet Archive.

Edit: For another full translation, look here. For other summaries of the dissertation, look here, or here. I did my own summary independently of these, and before reading them.

Koch uses char/min as the unit for speed, where a character is 10.2 dots. In modern convention, we use the length of “PARIS”, that is, 50 dot, including the word separation. So, divide the char/min value by 5 to convert into word-per-minute (wpm).

Work-Psychological Study of The Process of Receiving Morse Code

Or, a new training procedure for radio operators

A. Introduction

Work psychological aims at improving the effectiveness of workers by studying how the human mind adapts to the work procedures. The profession of radio operator is chosen since their main activity is that of receiving Morse signals, which can be easily studied and quantified.

B. General Considerations

Professional radio operators are required to perfectly understand Morse code at 100 chars/min. Even with automation, man remains indispensable in certain cases. Morse code encodes characters as a series of short elements, or “dots” (·), and long elements, or “dashes” (–) tones. In standard Morse code, the proportions are:

  • a: length of a dot
  • b: length of a dash, equal to 3 a
  • c: separation between two elements, equal to a
  • d: separation between two characters, equal to 3 a

In the human mind, a concept is not the sum of its parts, but an indivisible whole, or “gestalt” in German. The reader recognizes whole words instead of deciphering each letter. The same applies to Morse code.

C. Rhythm when Sending

We first study the effect of transmission speed on the psychological process. For this, experienced operators send the sequence “B C V Q F L H Y Z X” and the exact timings are recorded. Characters have different durations — E (·) takes 1 dot, but Ch (––––) 15 — so the speed is an average.

Subject 1Subject 2
Subject 3Subject 4

Average values of a, b, c, d for the 4 operators, at different speeds. Abscissa is the speed in chars/min and ordinate a, b, c and d, in a unit proportional to the duration.

At low speed, a and c are close (except in subject 4), as they should be. However, b is more than 3 a. More importantly, d is much longer than b, even though it should be equal to it. The longer pauses between the characters actually correspond to the fact that, between the pauses, characters are sent faster than the standard proportions would allow.

Subject 1Subject 2
Subject 3Subject 4

Lengths of a, b, c, d stacked. From the abscissa upwards, this would read as the letter “a”, including a letter pause at the end. The recorded timings are marked with “s”, and the standard timings as “o”.

Subject 4, who sent at standard proportions at low speeds, belongs to the optical learning type, and learned Morse from the visual symbols first. She can send reliably at a speed of 110-120 chars/min but cannot receive at more than 60-70 chars/min.

Thus, the operators adapt the proportions to keep the character “whole”.

Success rate of experienced operator receiving low-speed standard Morse code. The abscissa is the speed in chars/min and the ordinate is success rate in percent.

To verify this, an automatic device was set to send 30 characters at the standard proportions. Experienced operators had trouble receiving the characters at low speeds. So the “whole” effect appears at a speed of 50 chars/min.

Characters themselves are not sent regularly. The table below shows the difference between standard proportions, and the effective rhythm at slow speed.

CharacterStandardRecorded
L·–···  –··
F··–···–  ·
C–·–·–·  –·
Q––·–––  ·–
X–··––  ··–
Y–·––– ·––
K–·––  ·–
D–··–  ··

Similarly, the operator breaks the character down in subunits. In standard Morse, C is –·–·. But it is often broken into –· –· (dah dit … dah dit) or, more rarely, into –·– · (dah dit dah … dit). Other characters, such as S (···) cannot be broken down this way.

In summary, standard Morse code is only usable at a speed of 50 char/min or above.

D. Learning to Receive

I. Classical Training Procedures

a) The Analytical Method

First, the student memorizes the visual representation of the Morse code, sometimes with mnemonics. Then, they start training to receive auditory Morse code at a low speed, and gradually increase the cadence. The disadvantages of this method are:

  1. Having to learn the visual representation first
  2. Waiting for a character to finish
  3. No holistic effect at the beginning
  4. Having to switching from non-holistic to holistic sound patterns
  5. Having to switching from counting the dots and dashes to recognizing the sound pattern
  6. Adapting as the rhythm changes
b) The Sound Image method
Learning curve for the sound image method. The abscissa is the number of weeks of practice; the ordinate is the speed attained by the student.

Students learn the visual representation. When practicing reception, a longer pause is used between the individual characters, which are sent at a high speed from the beginning. Students use the pause between characters to think about what they just heard. As the speed increases, this becomes impossible, and they have to switch to immediate recognition. This shows as a plateau in the learning curve around a speed of 50 char/min. The disadvantage of this method are:

  1. Having to learn the visual representation first
  2. Thinking about the elements of the sound pattern during the long breaks.
  3. Having to switching from counting the dots and dashes to recognizing the sound pattern
  4. Adapting as the rhythm changes

II. A New Learning Procedure

To help the student directly recognize the sound pattern, the training should not teach the visual representation and should use a high speed from the start. Since the plateau is around 50 char/min, it should start with at least 60 char/min. Higher speeds make it hard to learn new characters, so just use 60 char/min.

Because of this higher starting speed, the training should start with just two characters. Other characters should be added one by one gradually as the previous ones are mastered. The speed is not increased until all characters are mastered.

The training starts with two very different characters. At first, the student is not told what the two characters are and just puts marks in the answer sheet:

This trains the student to recognize the sound pattern as a whole. Afterward, the characters corresponding to the sounds are named.

When writing the characters, a common mistake is to pause when not grasping a character immediately to try to think about it. However, this causes a cascading effects where the next characters are not grasped immediately either, or even entirely missed. To avoid this, the student should just write a dot when they do not grasp a character immediately.

The student should master these first two characters after about 10 minutes. Then, a letter is added, again without revealing which it is at first. Once the student accurately puts marks when this new character is sent, the character is revealed. This process is then repeated for each character. Once all characters are mastered, the speed can be increased.

Some characters are harder to learn. In general, X, Y, P, Q. They should be added in the last third of the training. Characters were added as L, F, C, K, R, P, X, Y, Q, D, B, G, A, N, Z, U, V, W, J, H, S, I, E, O, M, T:

Some Morse code characters are hard to distinguish, such as S (···) and H (····), U (··–) and V (···–), D (–··) and B (–···). At a higher speed, distinguishing characters composed of dash is easier: O (–––) and Ch (––––), G (––·) and 6 (–····), W (·––) and J (·–––). The characters V (···–) and B (–···) are too different to be confused. Starting the training with similar characters is not efficient: E (·), I (··), S (···), H (····).

Characters should be added gradually one by one. An experience was conducted where the characters were added several at a time: T, M, O, Ch, then E, I, S, H, then D, B, G, then U, V, W. Each addition reset the progress of the student.

Training sessions should be spread out in time instead of packed together, and should not exceed half an hour.

The experience was conducted with students, not with soldiers in training, so the schedule was not regular. In the first half-hour, trainers could learn up to 5 characters. Other characters are added more slowly. Before adding a new character, the error rate should be below 10 %. The duration of the training was limited to 24 to 28 half-hour sessions.

In addition, early practice can be helped by using a different pitch for the dots and for the dashes. The pitches are gradually brought together during the training. Merging the two pitches after the 15th or after the 20th letter makes no difference.

Learning curve for the single-pitch (dashed line) and two-pitch (solid line) methods. The abscissa shows the number of half-hour sessions, and the ordinate the number of characters mastered.

The characters were added as H, F, A, B, G, C, D, E, CH, L, K, I, O, T, M, P, Q, R, S, N, U, W, V, Y, X, Z (no J):

Plateaus corresponds to more difficult characters: P, X, V, Y, Q, Z. Starting the training with these characters in the first third of the training, but not at the very beginning, helps in ensuring they are mastered by the end of the training, without discouraging the student who just started.

Learning curve, with the hard characters added between the 10th and 13th half-hour of practice. The abscissa shows the number of half-hour sessions, and the ordinate the number of characters mastered.

13 to 14 year-old students have the same progression rate as adults.

Mastery of the alphabet at 60 char/min is reached in 14 hours with this method, against 7 weeks with the classical ones.

E. Aptitude Test

Those who fail to master 4 characters at a speed of 60 char/min in the first 2 half-hours never mastered all characters. So the beginning of the training also serves as an aptitude test before the candidate spent dozens of hours in training. Students should be grouped by aptitude to avoid slowing down the most apt. Some students could be impeded by slow writing, but this can be remediated with practice. Those accustomed to concentrating are best, in particular those of the acoustic imagination type. Among about 100 subjects, none of the diagnoses made on the basis of the first two half-hours led to a wrong judgment. Biegel does a similar aptitude test, but she starts at 40 char/min.

Categories
Amateur Radio

HamFox: Forking Firefox for Fun and no Profit

Null Cipher

The radio amateur service does not allow encryption of communications. This is a fundamental rule meant to ensure that the amateur service is not used for unintended purposes, such as commercial ones.

As a consequence, we cannot transmit passwords securely over the amateur frequencies. But we can use cryptographic authentication. Just not anything that would be used to hide the contents of the communications.

Cryptographic authentication is something that HTTPS/TLS allows out-of-the-box. We just need to make sure that we are not using encryption. In this article, we’ll see how to do this with Mozilla Firefox.

Forking Firefox

The code base of Firefox is an unwieldy Mercurial repository. Maintaining a fork would be cumbersome.

Git has won. And learning Mercurial for a small project adds quite a bit of friction, and I could not host on GitHub or GitLab. Of course, I can use git-remote-hg, but this adds another layer of abstraction, and another source of problems. I tried both, but it turned out it did not matter.

Since the official build process assumes that you are using the official repository with the official URLs and so forth, forking the repository itself would require also patching the build process. And I really did not want to touch that.

Another approach is to make a repository that just contains the patches to apply to the Firefox codebase, and a script that fetches the Firefox repository, applies the patches, and builds the project. This is gluon.

If you look into the HamFox repository, you will see that it contains just a few small files. The most important one is the null cipher patch itself.

The Patch

Grepping through the code for the TLS names of the cipher, we uncover an array sCipherPrefs that lists the supported ciphers. Here is an extract:

static const CipherPref sCipherPrefs[] = {

    {"security.ssl3.ecdhe_rsa_aes_128_gcm_sha256",
     TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
     StaticPrefs::security_ssl3_ecdhe_rsa_aes_128_gcm_sha256},

    {"security.ssl3.ecdhe_ecdsa_aes_128_gcm_sha256",
     TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
     StaticPrefs::security_ssl3_ecdhe_ecdsa_aes_128_gcm_sha256},

    //...
};

Each entry in the array is a CipherPref. The definition is given right above the array:

// Table of pref names and SSL cipher ID
typedef struct {
  const char* pref;
  int32_t id;
  bool (*prefGetter)();
} CipherPref;

The first field is the string used to toggle the cipher in the about:config page. There is no point is creating a toggle for this cipher, so we’ll just leave it empty.

The second field is the identifier of the cipher; we find that the macros are defined in security/nss/lib/ssl/sslproto.h. When looking for the null ciphers, we find:

$ grep NULL security/nss/lib/ssl/sslproto.h
#define SSL_NULL_WITH_NULL_NULL                TLS_NULL_WITH_NULL_NULL
#define SSL_RSA_WITH_NULL_MD5                  TLS_RSA_WITH_NULL_MD5
#define SSL_RSA_WITH_NULL_SHA                  TLS_RSA_WITH_NULL_SHA
#define TLS_NULL_WITH_NULL_NULL                 0x0000
#define TLS_RSA_WITH_NULL_MD5                   0x0001
#define TLS_RSA_WITH_NULL_SHA                   0x0002
#define TLS_RSA_WITH_NULL_SHA256                0x003B
#define TLS_ECDH_ECDSA_WITH_NULL_SHA            0xC001
#define TLS_ECDHE_ECDSA_WITH_NULL_SHA           0xC006
#define TLS_ECDH_RSA_WITH_NULL_SHA              0xC00B
#define TLS_ECDHE_RSA_WITH_NULL_SHA             0xC010
#define TLS_ECDH_anon_WITH_NULL_SHA             0xC015
#define SRTP_NULL_HMAC_SHA1_80                  0x0005
#define SRTP_NULL_HMAC_SHA1_32                  0x0006
#define SSL_FORTEZZA_DMS_WITH_NULL_SHA          0x001c

The SSL_* ones are just aliases for the TLS_* ones. We do want authentication. So, this eliminates TLS_NULL_WITH_NULL_NULL. We then have the choice between MD5, SHA-1 and SHA-256, and I opted for the last one, so TLS_RSA_WITH_NULL_SHA256.

The third one is a pointer to the function that tells us whether the toggle is enabled or not. There is no toggle. But we can just write a function that always returns true.

We replace the contents of the array with a single entry containing these fields, and we’re good!

To minimize the risk of conflict when the upstream code changes, I have put the other ciphers in a block comment instead of removing them altogether from the code. With this, we get:

diff --git a/security/manager/ssl/nsNSSComponent.cpp b/security/manager/ssl/nsNSSComponent.cpp
index 5844ffecfd..e2da79480a 100644
--- a/security/manager/ssl/nsNSSComponent.cpp
+++ b/security/manager/ssl/nsNSSComponent.cpp
@@ -1054,9 +1054,15 @@ typedef struct {
   bool (*prefGetter)();
 } CipherPref;
 
+bool AlwaysTrue(void) {
+  return true;
+}
+
 // Update the switch statement in AccumulateCipherSuite in nsNSSCallbacks.cpp
 // when you add/remove cipher suites here.
 static const CipherPref sCipherPrefs[] = {
+    {"", TLS_RSA_WITH_NULL_SHA256, AlwaysTrue},
+    /*
     {"security.ssl3.ecdhe_rsa_aes_128_gcm_sha256",
      TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
      StaticPrefs::security_ssl3_ecdhe_rsa_aes_128_gcm_sha256},
@@ -1103,6 +1109,7 @@ static const CipherPref sCipherPrefs[] = {
      StaticPrefs::security_ssl3_rsa_aes_128_sha},
     {"security.ssl3.rsa_aes_256_sha", TLS_RSA_WITH_AES_256_CBC_SHA,
      StaticPrefs::security_ssl3_rsa_aes_256_sha},
+    */
 };
 
 // These ciphersuites can only be enabled if deprecated versions of TLS are
Categories
Amateur Radio

HTTPS Without Encryption

As I have discussed previously, we can use cryptography over the amateur radio service, as long as we disable encryption. I have shown how to do this with SSH. Now, let’s see how to do this with HTTPS.

Modifying a Web browser, or even just a Web server, is an involved process, so we’ll start small, with the openssl command-line tool. On with crypto-crimes! No, not that crypto.

OpenSSL Server

The openssl command-line tool can be used like netcat to open a TLS session. To start a server, instead of:

$ nc -nlvp 4433
Listening on 0.0.0.0 4433

We’ll do:

$ openssl s_server
Could not open file or uri for loading server certificate private key from server.pem
4067D713C67F0000:error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file
4067D713C67F0000:error:80000002:system library:file_open:No such file or directory:../providers/implementations/storemgmt/file_store.c:267:calling stat(server.pem)

Uh oh.

Right. We’re doing TLS, so we’ll need some cryptographic key.

If you are on a Debian-based Linux distribution (e.g. Ubuntu), you can always use the “snakeoil” key:

$ sudo openssl s_server -key /etc/ssl/private/ssl-cert-snakeoil.key -cert /etc/ssl/certs/ssl-cert-snakeoil.pem -www
Using default temp DH parameters
ACCEPT

Note that we need to run the command as root to be able to read the private key. The -www option starts a basic Web server behind the TLS server, so you can test the setup by browsing to https://127.0.0.1:4433/. It will serve a Web page giving you the command-line you use as well as debugging information about the TLS connection.

Note: your Web browser does not know the emitter of the cryptographic key, so it will show a scary warning as shown below. This is expected, so you can just click on “Advanced…” and then on “Accept the Risk and Continue”.

Never trust the people at 127.0.0.1.

We’ll want to create our own cryptographic key instead of using the “snakeoil” one. Also, we’ll need a corresponding “.pem” file, or certificate, that will give a name (“CN” for “Common Name”) to this key. This is done with the following commands.

$ openssl genrsa -out server.key 4096
$ openssl req -new -x509 -key server.key -out server.pem -subj "/CN=localhost"

Once this is done, we can start a TLS server with:

$ openssl s_server -key server.key -cert server.pem -www

As with the “snakeoil” certificate, you can check the connection from your Web browser:

The warning sign on the padlock in the address bar is telling you that the connection might not be fully secure. Do not worry. It is going to become worse. Much worse.

OpenSSL Client

We’ve seen nc -nlvp 4433. Now, let’s see nc -v 127.0.0.1 4433:

openssl s_client

By default, it will try to connect to port 4433 on the local loopback. If you have a running s_server without the -www option in another terminal, you will be able to send messages from one another.

Note: if you do not remove the -www option from the s_server command, you will not see your inputs as you might expect, since they will be either ignored (server side) or sent as part of the HTTP protocol (client side).

Note: input is sent line-by-line, so you won’t see your message in the other terminal until you hit Return.

If you want to feel like a leet-h4x0r, you can use the s_client command to connect to any public TLS server. For instance, instead of clicking on the link https://perdu.com, you may run the following command

$ openssl s_client -connect perdu.com:443

Then, type the following and hit Return twice (you need to send a blank line).

GET / HTTP/1.1
Host: perdu.com

You should see the HTML for the Web page being printed in your terminal:

<html><head><title>Vous Etes Perdu ?</title></head><body><h1>Perdu sur l'Internet ?</h1><h2>Pas de panique, on va vous aider</h2><strong><pre>    * <----- vous &ecirc;tes ici</pre></strong></body></html>

Disabling Encryption

What we’ve done so far is basically the equivalent of the code shown below, except everything was encrypted.

$ nc -nvlp 4433
$ nc -v 127.0.0.1 4433
$ nc perdu.com 80
GET / HTTP/1.1
Host: perdu.com

To convince ourselves of it, we can use Wireshark. Once it’s started, select the any interface, type tcp.port==4433 as the filter, and hit Return.

Mind like a blank slate. Like a baby. Or me when I have to read reduced standard C++ headers.

Oh! And go play with the s_server and s_client command like we’ve seen above. You should see stuff like this:

Seriously, Wireshark is cool. It knows stuff.

This shows network packets being exchanged on the TCP port 4433 (the one used by default by s_server and s_client). The one I have selected corresponds to the transmission of “Hello World” from the client to the server. You can see it. It’s right here in the “Encrypted Application Data”. Okay, you cannot really see it, since it’s encrypted, by take my word for it.

Now that we have done all of that, we can ask ourselves the fun question: how can we make all of this pointless?

Let’s disable this all “confidentiality-preserving” technobabble “encryption” stuff.

First, let’s ask openssl to tell it what encryption modes it supports. On my computer, I get:

$ openssl ciphers -s -v
TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 Kx=any Au=any Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH Au=RSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
DHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=DH Au=RSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA384
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA384
DHE-RSA-AES256-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AES(256) Mac=SHA256
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA256
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA256
DHE-RSA-AES128-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AES(128) Mac=SHA256
ECDHE-ECDSA-AES256-SHA TLSv1 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA1
ECDHE-RSA-AES256-SHA TLSv1 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA1
DHE-RSA-AES256-SHA SSLv3 Kx=DH Au=RSA Enc=AES(256) Mac=SHA1
ECDHE-ECDSA-AES128-SHA TLSv1 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA1
ECDHE-RSA-AES128-SHA TLSv1 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA1
DHE-RSA-AES128-SHA SSLv3 Kx=DH Au=RSA Enc=AES(128) Mac=SHA1
AES256-GCM-SHA384 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(256) Mac=AEAD
AES128-GCM-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(128) Mac=AEAD
AES256-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA256
AES128-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA256
AES256-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA1
AES128-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA1

Each line is a possible setting (“cipher suite”). What matters is the Enc= part. All of these settings are using some mode of “AES” as encryption. “AES” stands for “Advanced Encryption Standard”. Booooring.

We want no security. None. Zilch. Seclevel zero.

$ openssl ciphers -s -v 'NULL:@SECLEVEL=0' 
TLS_AES_256_GCM_SHA384         TLSv1.3 Kx=any      Au=any   Enc=AESGCM(256)            Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256   TLSv1.3 Kx=any      Au=any   Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256         TLSv1.3 Kx=any      Au=any   Enc=AESGCM(128)            Mac=AEAD
ECDHE-ECDSA-NULL-SHA           TLSv1   Kx=ECDH     Au=ECDSA Enc=None                   Mac=SHA1
ECDHE-RSA-NULL-SHA             TLSv1   Kx=ECDH     Au=RSA   Enc=None                   Mac=SHA1
AECDH-NULL-SHA                 TLSv1   Kx=ECDH     Au=None  Enc=None                   Mac=SHA1
NULL-SHA256                    TLSv1.2 Kx=RSA      Au=RSA   Enc=None                   Mac=SHA256
NULL-SHA                       SSLv3   Kx=RSA      Au=RSA   Enc=None                   Mac=SHA1
NULL-MD5                       SSLv3   Kx=RSA      Au=RSA   Enc=None                   Mac=MD5 

That’s more like it! We have several options to choose from. You see these Mac=MD5? I know we are trying to give a syncope to every security researcher on the planet, but we still respect ourselves. We’ll use NULL-SHA256. It’s the most secure of the totally insecure cipher suites. Trust me, I have a PhD in cryptography.

Let’s redo the s_server and s_client stuff again, with this cipher suite.

$ openssl s_server -key server.key -cert server.pem -cipher NULL-SHA256:@SECLEVEL=0 -tls1_2
$ openssl s_client -cipher NULL-SHA256:@SECLEVEL=0
Hello World

Note: we need to add the -tls1_2 option; otherwise, OpenSSL will offer TLS 1.3. The cipher suite we have selected is only available for TLS 1.2, and OpenSSL uses a separate option, -ciphersuites (yes, seriously) to configure TLS 1.3 cipher suites. Just add -tls1_2 and don’t think about it too much.

Hey, did you notice we were doing IPv6? That’s totally transparent, thanks to the magic of the OSI model. Wait no, TCP/IP is not actually OSI. Also, TCP and IP are not independent. Also learning the OSI model will probably never be useful to you. No, that had nothing to do with the contents of this article. Mostly.

Totally different, you see?

Actually, yes. 48656c6c6f20576f726c64 is the hexadecimal representation of the ASCII encoding of Hello World:

Who can’t read ASCII fluently?

Great! Now, we’re really back at Netcat-level security.

Note: sure, the client is checking the certificate of the server. And we could do all that with a certificate signed by Let’s Encrypt so that it’s not actually fully useless. Oh yeah, right, no, the Web browser is all panicky about the “no security” thing and won’t even let you enable the NULL-SHA256 cipher.

The error messages are getting worse. It means we are going in the right direction.

Client Authentication

Okay, okay, wait up. There is one more thing we can do with this, and it might serve some kind of purpose. The browser is checking the authenticity of the server. But what if, like, the server checked the authenticity of the client, man?

Yes, we can do this!

Modern Web browsers are a bit fussy when we disable encryption, so the command-lines in this section will not do so. That way, you can try in your Web browser. But, know that you can combine with the NULL-cipher, and test with the s_client command.

First, let’s tell the server to check the authenticity of the client. We tell it to use server.pem as the authority for client certificates. That is, to be considered valid, client certificates must have been signed by the private key of the server.

$ openssl s_server -key server.key -cert server.pem -verify 1 -CAfile server.pem
verify depth is 1
Using default temp DH parameters
ACCEPT

Then, let’s create a certificate for the client. Since it should be signed by the server’s key, that’s what we do.

$ openssl genrsa -out client.key 4096
$ openssl req -new -key client.key -out client.csr -subj '/CN=John Doe'
$ openssl x509 -req -CA server.pem -CAkey server.key -in client.csr -out client.pem
$ openssl pkcs12 -export -out client.p12 -in client.pem -inkey client.key -passout pass:

Note: the last one is only useful if you want to import the client certificate in a Web browser.

And now, we can use it to authenticate:

$ openssl s_client -cert client.pem -key client.key

On the server’s side, you should see something like:

$ openssl s_server -key server.key -cert server.pem -verify 1 -CAfile server.pem
verify depth is 1
Using default temp DH parameters
ACCEPT
depth=1 CN = localhost
verify return:1
depth=0 CN = John Doe
verify return:1
…
subject=CN = John Doe
issuer=CN = localhost

Note: alternatively, you can add the -www option to s_server and navigate to https://127.0.0.1:4433/. At the bottom, you’ll see no client certificate available. You can change that by loading the client certificate in your Web browser.

Note: to add the certificate in Firefox, go to:

  1. “Settings”
  2. “Privacy & Security”
  3. “Certificates”
  4. “View Certificates”
  5. Your Certificates”
  6. “Import…”
  7. Select the client.p12 file
  8. Give an empty password

Okay, but… Why?

Good question! When we do this, we are allowing the client to authenticate using a secret file on the computer (the private key). This could be used in an actual Web service to let people connect to their account without using a password. That’s a good thing. Since we’re going to disable encryption, we better not have passwords flying on the air!

Conclusion

Now, all of this was a bit academic. But it has allowed us to test out the waters of disabling encryption and using client authentication. It’s also going to be useful when we want to check that a modified Web browser can connect to an s_server instance with NULL encryption, or that we can authenticate to an HTTPS service with a client certificate.

But this will be the topics for other posts!

Categories
Amateur Radio

HamSSH

In a previous article, I explained that encryption is not allowed on the radio amateur service, but that we could still use public-key cryptography. In particular, we could use SSH for authentication, but without encryption. This would allow secure remote control of base stations, satellites and more.

In this article, I will discuss how we can do that in practice without reinventing the wheel. The result is the HamSSH project, which forks OpenSSH to provide an encryption-free, but still authentication-secure implementation of SSH.

Building OpenSSH

The most common implementation of the SSH protocol is OpenSSH. To build the project from the sources, we run the commands below:

$ sudo apt install autoconf gcc git libssl-dev make zlib1g-dev
$ git clone https://github.com/openssh/openssh-portable
$ cd openssh-portable
$ autoreconf
$ ./configure
$ make -j$(nproc)

Once this is done, we can generate a server key with ssh-keygen, start the server and try connecting with the client. You should be able to do that by running the commands below in two terminals:

# Terminal 1
$ ssh-keygen -f host_key -N ''
$ $PWD/sshd -d -f none -o "HostKey $PWD/host_key" -o "Port 2222

# Terminal 2
$ ./ssh -p 2222 localhost

Enabling the Null-Cipher

You may try using -o "Ciphers none" on the server’s command-line, and -c none on the client’s, but it will fail because the null-cipher is not enabled. Our first task is to find how to make it available.

Thanks to the -d option on the server’s command-line, we can check what encryption cipher have been used:

debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com [preauth]
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com [preauth]

From this, we can take a guess and search for the string chacha20 to locate the place where the ciphers are chosen. We quickly find an array named ciphers, in cipher.c. It looks promising:

static const struct sshcipher ciphers[] = {
#ifdef WITH_OPENSSL
#ifndef OPENSSL_NO_DES
	{ "3des-cbc",		8, 24, 0, 0, CFLAG_CBC, EVP_des_ede3_cbc },
#endif
	{ "aes128-cbc",		16, 16, 0, 0, CFLAG_CBC, EVP_aes_128_cbc },
	{ "aes192-cbc",		16, 24, 0, 0, CFLAG_CBC, EVP_aes_192_cbc },
	{ "aes256-cbc",		16, 32, 0, 0, CFLAG_CBC, EVP_aes_256_cbc },
	{ "aes128-ctr",		16, 16, 0, 0, 0, EVP_aes_128_ctr },
	{ "aes192-ctr",		16, 24, 0, 0, 0, EVP_aes_192_ctr },
	{ "aes256-ctr",		16, 32, 0, 0, 0, EVP_aes_256_ctr },
	{ "aes128-gcm@openssh.com",
				16, 16, 12, 16, 0, EVP_aes_128_gcm },
	{ "aes256-gcm@openssh.com",
				16, 32, 12, 16, 0, EVP_aes_256_gcm },
#else
	{ "aes128-ctr",		16, 16, 0, 0, CFLAG_AESCTR, NULL },
	{ "aes192-ctr",		16, 24, 0, 0, CFLAG_AESCTR, NULL },
	{ "aes256-ctr",		16, 32, 0, 0, CFLAG_AESCTR, NULL },
#endif
	{ "chacha20-poly1305@openssh.com",
				8, 64, 0, 16, CFLAG_CHACHAPOLY, NULL },
	{ "none",		8, 0, 0, 0, CFLAG_NONE, NULL },

	{ NULL,			0, 0, 0, 0, 0, NULL }
};

We can see that the none cipher is actually listed. Now, we need to find out how to allow it. The sshcipher struct that make the elements of the array is defined right before:

struct sshcipher {
	char	*name;
	u_int	block_size;
	u_int	key_len;
	u_int	iv_len;		/* defaults to block_size */
	u_int	auth_len;
	u_int	flags;
#define CFLAG_CBC		(1<<0)
#define CFLAG_CHACHAPOLY	(1<<1)
#define CFLAG_AESCTR		(1<<2)
#define CFLAG_NONE		(1<<3)
#define CFLAG_INTERNAL		CFLAG_NONE /* Don't use "none" for packets */
#ifdef WITH_OPENSSL
	const EVP_CIPHER	*(*evptype)(void);
#else
	void	*ignored;
#endif
};

The name field is self-explanatory. The fields block_size, key_len, iv_len and auth_len are parameters for the cipher. They are not used by the null-cipher, so they are set to zero in ciphers. The last field, evptype or ignored looks optional, so it probably does not matter for what we want to do.

That leaves flags. Given the definition of CFLAG_INTERNAL and the associated comment, it seems reasonable to think that this macro controls what ciphers should not be made available in normal operation.

If we look further, we find two functions that use this macro:

/* Returns a comma-separated list of supported ciphers. */
char *
cipher_alg_list(char sep, int auth_only)
{
	char *tmp, *ret = NULL;
	size_t nlen, rlen = 0;
	const struct sshcipher *c;

	for (c = ciphers; c->name != NULL; c++) {
		if ((c->flags & CFLAG_INTERNAL) != 0)
			continue;
		if (auth_only && c->auth_len == 0)
			continue;
		if (ret != NULL)
			ret[rlen++] = sep;
		nlen = strlen(c->name);
		if ((tmp = realloc(ret, rlen + nlen + 2)) == NULL) {
			free(ret);
			return NULL;
		}
		ret = tmp;
		memcpy(ret + rlen, c->name, nlen + 1);
		rlen += nlen;
	}
	return ret;
}
…
#define	CIPHER_SEP	","
int
ciphers_valid(const char *names)
{
	const struct sshcipher *c;
	char *cipher_list, *cp;
	char *p;

	if (names == NULL || strcmp(names, "") == 0)
		return 0;
	if ((cipher_list = cp = strdup(names)) == NULL)
		return 0;
	for ((p = strsep(&cp, CIPHER_SEP)); p && *p != '\0';
	    (p = strsep(&cp, CIPHER_SEP))) {
		c = cipher_by_name(p);
		if (c == NULL || (c->flags & CFLAG_INTERNAL) != 0) {
			free(cipher_list);
			return 0;
		}
	}
	free(cipher_list);
	return 1;
}

The code is pretty straightforward. The first function iterates over the list of ciphers, and filters out those whose flags field matches the CFLAG_INTERNAL macro. The second one takes a comma-separated list of cipher names, and check that they all correspond to known ciphers in the list, and that they are not internal ciphers.

If we change CFLAG_INTERNAL to 0, that should enable none as a normal cipher.

Note: you might want to set CFLAG_INTERNAL to ~CFLAG_NONE to disable the real ciphers, but this just makes sshd segfault. This is because the default list of ciphers becomes invalid. We’ll get back tot his.

After rebuilding, restarting the SSH server, and connecting with the client, we get:

debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com [preauth]
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com [preauth]

We’re still using encryption!

It’s actually normal: we enabled the null-cipher, but it’s not necessarily selected. In fact, it is still not listed as one of the options by either the server of the client. We can force this by adding -o "Ciphers none" to the server’s command-line, and -o none to the client’s. Then, we get:

debug1: kex: client->server cipher: none MAC: umac-64-etm@openssh.com compression: zlib@openssh.com [preauth]
debug1: kex: server->client cipher: none MAC: umac-64-etm@openssh.com compression: zlib@openssh.com [preauth]

Success!

But we want this to be the default behavior. In fact, we would like to ensure we do not use encryption by mistake, so we want to disable the other encryption modes.

Disabling Encryption

Let’s look into how the option Ciphers is handled

If we look in the other source files containing the string chacha20, we find this in myproposal.h:

#define	KEX_SERVER_ENCRYPT \
	"chacha20-poly1305@openssh.com," \
	"aes128-ctr,aes192-ctr,aes256-ctr," \
	"aes128-gcm@openssh.com,aes256-gcm@openssh.com"

#define KEX_CLIENT_ENCRYPT KEX_SERVER_ENCRYPT

“KEX” stands for “KEy eXchange”. So KEY_…_ENCRYPT relates to the stage where the server and the client decide how to encrypt the communications. The macro is used in the servconf.c and readconf.c source files, where the server and the client (respectively) detect the default values for the Ciphers option. Let’s try changing the string to just "none", and starting the server and the client without any particular option.

debug1: kex: client->server cipher: none MAC: umac-64-etm@openssh.com compression: zlib@openssh.com [preauth]
debug1: kex: server->client cipher: none MAC: umac-64-etm@openssh.com compression: zlib@openssh.com [preauth]

Good!

But what if we pass -o "Ciphers chacha20-poly1305@openssh.com" to the server and -c chacha20-poly1305@openssh.com to the client?

debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com [preauth]
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com [preauth]

Let’s make sure this does not happen. We can now come back to the definition of CFLAG_INTERNAL and set it to ~CFLAG_NONE. Since the default cipher list is "none", this will not make the server crash. And if we try to use anything but null-cipher, we get:

// server
Bad SSH2 cipher spec 'chacha20-poly1305@openssh.com'.
// client
Unknown cipher type 'chacha20-poly1305@openssh.com'

Done!

Niceties

We now have a null-cipher-only implementation of an SSH server and client. Let’s see what we can do further to avoid mistakes.

Changing the Names

First, let’s make sure we do not confuse the regular SSH binary, and our null-cipher only one. We’ll also want to avoid overwriting the regular ones by mistakes, and we might want to use different configurations files.

HamSSH prepends a “ham” suffix to all the binary and configurations files. So the user configurations files can be found in ~/.hamssh, and the global configurations files are in /usr/local/etc/ham*.

Changing the Default Port

Regular SSH uses the port 22 by default. We will want to use another one for our null-cipher-only SSH. Since 21 is already taken by FTP, I opted for 23. It is normally used for Telnet. Since Telnet is even more obsolete than FTP, that should not be a problem. And users can always use another port by using the regular SSH options.

I also like the idea that it reminds us that all communications are in the clear, just like with Telnet.

Disabling Password Authentication

The SSH protocol is still useful without encryption because it gives us public-key encryption. But we do not want to risk the user mistakenly typing a password when trying to log in.

In other words, we want to force the PasswordAuthentication and KbdInteractiveAuthentication options to No. The simplest way is to overwrite options.password_authentication options.kbd_interactive_authentication after the command-line arguments and configuration files have been parsed, in sshd.c (server) and ssh.c (client):

	/* Force public key authentication */
	options.password_authentication = 0;
	options.kbd_interactive_authentication = 0;

Conclusion

With just a few tweaks, we have a robust and versatile tool for remote control compatible with amateur radio regulations. We have forcibly disabled encryption on both the server and the client to avoid using encryption by mistake, disabled password authentication to avoid spilling secrets carelessly, and changed names and the default port for compatibility with regular SSH. All we need to do now is to use it!

Categories
Amateur Radio

Ham Crypto

Amateur Radio

Amateur radio is about tinkering with the electromagnetic field. This may involve designing analog circuits to shape or unveil the modulation of signals, or sometimes using digital ones to recover information from the noise, or operating multi-hundred-watts transceivers to chat directly with the other side of the world, or lower-power ones to go just as far with Morse code, or even bouncing signals off the Moon. Also, lasers.

Yes, this is the 2005 picture from Wikipedia. No, things have not changed that much since. Physics still work the same.

Amateur radio operators, or “hams” for short, benefit from special privileges to this end. The radio spectrum is a crowded place; yet hams are allowed to transmit at various frequencies spread between 135,7 kHz and 250 GHz (not all bands are visible in the chart). Radio equipment is normally strictly regulated to limit interference and operations in unauthorized bands; yet hams are allowed to modify transmitters as they please, or even build and operate their own from scratch.

The allocation of the radio spectrum in the United States. The allocation is similar in other countries. You might notice there is not much free room. Click to enlarge.

It would be very tempting for a commercial operator, for instance, to make use of these privileges for their own benefit. To avoid this, there is one very important restriction: all communications must be done in the clear.

Code is Law

The exact wording from the international regulations is:

Transmissions between amateur stations of different countries shall not be encoded for the purpose of obscuring their meaning

Article 25, paragraph 2A from the Radio Regulations edited by the ITU (emphasis mine)
(yes, the website is very slow, which is a bit sad for the International Telecommunication Union)

Note: you might think that “of different countries” means that hams can encrypt their communications within a country, and that can actually be the case. However, the article is meant to bind countries to a common standard, not to restrict what happens within each country. In practice, each country will decide whether they want to allow it locally. And, in general, they don’t.

Encoding is the mapping of information to a particular representation. For instance, Morse Code and ASCII are encodings that map the letters of the alphabet to short and long signals, and to numbers, respectively. AM and FM are encodings that map analog signals to other analog signals, that is to say modulation.

Encoding is necessary for any form of communication, so it is of course allowed in general. What is forbidden is to represent information in a way that hides its meaning.

The ASCII encoding maps the uppercase and lowercase Latin letters, as well as the Arabic numerals, some punctuation marks, and control characters

Note that the wording does not say “cryptography”. This is intentional. Cryptography is just a set of techniques which can be used to hide information. It does not have to be this way, and this is not the only way to do so. This is about intent.

If you are doing Morse Code, SSB, FT8, or experimenting with your own super-advanced error-correction encoding that is meant to be usable by anyone, you would be in the spirit of the law. On the opposite, if you are hiding information in the low bits of a digital image, or using a custom protocol only you and your buddies know about, or using a proprietary protocol whose specification is not public, you are going against the spirit of the law.

Steganography is the idea of hiding information in plain sight

Note that the difference between “experimenting with a new protocol” and “designing a protocol restricted to just a few people” is slim. What matters is intent. A way to show good faith would be to include a link (using plain voice or Morse code) to an open source repository that anyone can easily install to decode the rest of the communication. To be clear, that’s not an original take.

What about cryptography? It should obviously be banned, right? The entire point of this discipline is to hide information.

My plant in the audience to ask a convenient question at a convenient time for my transition

Mostly, yes. But we are here to tinker, so let us see what this “mostly” entails.

Confidentiality and Authentication

The two main use cases of cryptography are confidentiality and authentication. Mostly everything else derives from these.

They are mostly used together. For instance, the most widespread implementation of cryptography is indubitably TLS, the protocol used to secure all communications of the Web. This the S in HTTPS. This is also what enabled online shopping, and grew the Web from a niche protocol to visit personal homepages to the societal backbone it is today.

The green padlock is telling you that the communications with the website are made secure using cryptography. It does not tell you that the website itself is trustworthy, however.

Nowadays, when you connect to a website, your Web browser will ensure two things.

  1. Communications are encrypted, so that no one snooping around can read them. That’s the “confidentiality” part.
  2. You are talking to the right website. Otherwise, it would be pretty easy to intercept all your communications and read them. This is how company proxies can monitor what you are doing. That’s the “authentication” part.

Note that authentication is necessary if you want confidentiality. Without at least some basic form of authentication (e.g. sharing a private key), encryption will only give confidentiality against passive attack models.

However, authentication on its own could definitely make sense. And the purpose of authentication is not to obscure the meaning. In particular, a cryptographic signature is a piece of data that anyone can decode and interpret. The information it contains is simply the fact that the associated message was indeed sent by its author, not by someone else.

Note: the usual rebuttal is this would allow an attacker to send the same message again. However, this is a well-known topic and easily solvable.

To be clear, I am not the first person to think about doing this. Nor the second:

But, why?

Why not? The purpose of amateur radio is to experiment with communication technology. Knowledge can be an end in itself. But still, having some ideas of what practical applications could look like can be motivating, so I list some here.

The Glider, the smallest moving pattern in Conway’s Game of Life

Remote Control. Since 2004, Radio Regulations include an exception to article 25.2A for the control of satellites. Indeed, going to orbit is hard¹, so you want to be able to service satellites remotely. But if anyone can remotely turn off your satellite, or make it dive into the atmosphere, you’ve got a problem. So you need some way to authenticate the commands. But you do not actually need encryption. This means that remote control over amateur radio is possible for things other than satellites. Remote base stations, or repeaters, for instance.

¹ In fact, low Earth orbit is halfway to anywhere in the Solar System.

Yes, people are actually building satellites, and actually sending them to space

Authenticated QSOs. Such schemes could be used to authenticate contacts, or QSOs, to prevent people from spoofing call signs. This could serve a similar purpose as PGP or GPG for email.

Contesting. Possibly the most popular activity among hams is contesting. That is, contacting as many other operators in the world in a given time frame. Nowadays, scoring is done by having all the operators send the log of their contacts to the contest’s website. If K1AA says they have contacted F6ZZ at 28,500 MHz around 14:00 UTC, and F6ZZ says they have contacted K1AA at the same frequency at around the same time, we can probably assume that this actually happened. Unless K1AA and F6ZZ are colluding, of course, but this is hard to prevent. But if, for some reason, F6ZZ is unable, or forgets, to upload their logs, then the contest will assume that K1AA made a mistake, and subtract points. With authenticated QSOs, K1AA could provide strong evidence that the contacts happened as described. Contests already routinely require the exchange of serial numbers, so it might not be such a stretch to implement. Instead of an incrementing number, each operator might have a computer generate a short code for each contact.

Wow. Much radio. So frequency. Many QSOs.

Online services. Now that digital modes and packet communications are finally being used in amateur radio, it is possible to use protocols such as HTTP. But, without authentication, the use cases are as limited as the Web in the early 1990s. With authentication, we could envision message boards and editable wikis with secure user accounts.

These are just the few examples I could think of, but there might be other relevant applications. Of course, there are wrinkles to be ironed out, but I do not pretend to have it all figured out.

Null Cipher

How do we use that in practice? We could roll our own crypto, or not. On the one hand, it is deceptively easy to make mistakes when implementing cryptography by oneself. On the other hand, the purpose of amateur radio is to experiment. So we might play around with custom cryptographic protocols over the air, but maybe we should use proven ones for actual persistent use. Let’s see what we could use for this.

TLS. The first obvious candidate is the omnipresent TLS. It is mostly used to authenticate servers, but you can also use it to authenticate clients. This has already been discussed in the past in the context of amateur radio to authenticate clients for remote base stations. Using TLS with the “NULL” cipher is a common topic, and it is not very hard to tell OpenSSL to do so (-cipher NULL-SHA256:@SECLEVEL=0). Modifying some web server to serve content with the NULL cipher is more involved, but probably not that hard.

However, both Mozilla Firefox and Google Chrome have removed the NULL cipher from the list of supported ciphers. In other words, they will refuse to connect without encryption. Fixing this would involve modifying their source code (easy), navigating their complicated build process (annoying), distributing them to the users (hard), and maintaining the fork (who wants to do that anyway?). Unfortunately, this is probably necessary, if we also want to prevent users from inadvertently using encryption by connecting to HTTPS websites. I’ll probably look into it at some point, but this is a significant project.

Me writing some random text through an SSH connection

SSH. Another candidate is SSH, which is already widely used for remote control of virtually everything nowadays (yes, I know, everything is terrible). It authenticates both the server and the client. The source code and the build process are also fairly simple, so it is easy to make a server and a client that can talk to each other without encryption. The nice thing with SSH is that you can then redirect ports through it, use it as a SOCKS proxy, or even as a VPN.

Wireshark intercepting the raw communications for the SSH sessions over the network. As you can see, the text is visible in clear. The [0m, [1m, [27m and so on are ANSI escape sequences. The rest are the SSH handshake and the cryptographic signature, which are easier to read when displayed as such.

Conclusion

To sum up, I like the idea of using cryptographic tools to provide proper authentication for remote control and enable new use cases through amateur radio. Of course, there is significant work to be done before we can get to there, but it’s nice to know that we do not need to start from scratch.