Tales from the jar side: Property-based testing the Caesar Cipher, ST:TMP out in 4K, and Good tweets

My wife asked me to put ketchup on the shopping list, but now I can't read it

Apr 10, 2022

Welcome, fellow jarheads, to Tales from the jar side, the Kousen IT newsletter, for the week of April 3 - April 10, 2022. This week I taught the first week of my Spring and Spring Boot in 3 Weeks course and my Managing Your Manager course on the O’Reilly Learning Platform, and my Spring MVC course as an NFJS Virtual Workshop.

Regular readers of this newsletter are affectionately known as jarheads, and are far more intelligent, sophisticated, and attractive than the average newsletter reader. If you wish to become a jarhead, please subscribe using this button:

If any of my newsletters run long (as this one did), the easiest way to see the whole thing is to click on the title. That will open the post in a browser, formatted correctly even if you’re on mobile. I can’t believe it took me this long to find out about that. It’s extremely helpful.

Again:

Click On The Title for long newsletters distributed from Substack, like this one

Property-based Testing and Julius Caesar

Last week I mentioned I proposed a couple of talks for this year’s NFJS tour involving subjects I didn’t know very well. I called those talks ambitious, because while I was writing my newsletter I completely blanked on the actual term I use. That term is aspirational — I proposed the talks because I aspire to understand the topics better. I don’t know why I couldn’t think of that word last week, but at least I remember it now.

One of my aspirational talks this year is about property-based testing (PBT). PBT is about specifying properties — features that should be true about an object at all times. An example is if your program encodes a string, then decoding it should give back the original string. The idea of encoding and then decoding — i.e., doing an operation and then doing its reverse — is a common way of finding properties. PBT tools generate tests for you, and check all the properties for each.

I must admit that while I am a fan of testing (mostly because I don’t believe my code is right until I test it), I don’t enjoy finding problems in my own code. On the other hand, I do like testing other people’s code, mostly because that way I know whether or not I believe it.

One such example comes from the website Baeldung, however that is pronounced. That site contains short, self-contained tutorials on a wide variety of subjects related to Java, so you’d think I’d be a big fan. I mostly am, but the problem with those tutorials is that they all contain embedded advertisements, some of which make the site perform poorly. Also, they are relentless about pushing their own courses. I understand that, in the sense that something has to fund all that work putting together the tutorials, but it makes me reluctant to visit the site.

As I was getting into PBT, however, I asked my friend and fellow NFJS speaker Dan Hinojosa (Hi Dan!) what example he liked for demos, and he mentioned the Caesar Cipher.

If you’ve never heard of it, the story is that Julius Caesar needed to communicate with his generals during his campaigns in Gaul, to send an encoded message like oppugnare prima luce (Latin for “attack at dawn,” according to Google Translate). The technique he used was to shift all the letters in the alphabet by a certain number of characters and write down the corresponding letters. Wikipedia (linked above) has a trivial example of a left-shift by 3 characters:

That’s a very easy code to break, but since Caesar was dealing with a largely illiterate populace, it was considered good enough at the time. The Romans even made up wheels to do the encoding and decoding for you by rotation:

It’s not hard to do this programmatically, which brings me back to the Baeldung site. They have an article called The Caesar Cipher in Java that includes code on how to do the encryption and decryption, as well as how to break encoded messages.

(The “how to break it” part feels unnecessarily complicated to me. I mean, if you’re trying to decode a message and there are only 26 possible values for the shift in English, just try them all and see which one contains readable words. But I digress.)

As with most of their articles, the source code is located in a GitHub repository, which also includes a few simple tests. I decided to grab the code and their tests and put them through the PBT approach.

For the record, here are the cipher and decipher methods included in their CaesarCipher class:

public String cipher(String message, int offset) {
  StringBuilder result = new StringBuilder();

  for (char character : message.toCharArray()) {
    if (character != ' ') {
      int originalAlphabetPosition = 
        character - LETTER_A;
      int newAlphabetPosition = 
        (originalAlphabetPosition + offset) % ALPHABET_SIZE;
      char newCharacter = 
        (char) (LETTER_A + newAlphabetPosition);
      result.append(newCharacter);
    } else {
      result.append(character);
    }
  }

  return result.toString();
}

public String decipher(String message, int offset) {
  return cipher(message, 
    ALPHABET_SIZE - (offset % ALPHABET_SIZE));
}

The decipher method just calls the cipher method with the reverse of the offset.

Their test class, CaesarCipherUnitTest, checked their code for a shift of 3, a shift of 10, and a shift of 36 (which wraps around by 10), all for the same string. From a PBT point of view the obvious question is, how many tests of specific shifts is enough? How many strings do you need to use? How many tests do you need to be confident the code is correct?

Here is my initial PBT test, using the jqwik library created by Johannes Link:

@Property
void checkAllShifts(@ForAll @NotBlank String message,
                    @ForAll @Positive int offset) {
  String encoded = algorithm.cipher(message, offset);
  assertThat(algorithm.decipher(encoded, offset))
    .isEqualTo(message);
}

The annotations tell the framework to check all strings that have at least one non-blank character, for all positive offsets. This fails rather spectacularly:

Shrunk Sample (12 steps)
------------------------
  arg0: "!"
  arg1: 1

Original Sample
---------------
  arg0: "坘⓼㞲鐍ँ竕㪡䚨役濫⧤臐ፙ൹늶콉䵌ᤳ"
  arg1: 10068

  Original Error
  --------------
  org.opentest4j.AssertionFailedError:
    expected: "坘⓼㞲鐍ँ竕㪡䚨役濫⧤臐ፙ൹늶콉䵌ᤳ"
     but was: "hlraysozigtluyxgjk"

When jqwik says all strings, it means all strings. I have no idea what most of those characters are, but when the framework shrank the error to its minimum possibility, it immediately hit a problem where the string was just “!” and you shifted by 1.

As it turns out, that was my fault. If you read the actual blog post (probably a good idea), it contains the phrase “suppose here that offsets are positive and messages only contain lower case letters and spaces,” so I should only be trying out lowercase letters. So I added the annotations @LowerCase and @Chars({' '}), in order to restrict the strings to lowercase letters and spaces. This, too, fails:

Shrunk Sample (7 steps)
-----------------------
  arg0: "b"
  arg1: 2147483647

The problem is the shift, which in this case is the maximum integer value in Java. Java integers range from -2^31 to 2^31 - 1, which is not symmetrical because it includes zero. So going to the maximum integer meant it couldn’t decode, because the the shift plus the integer value for “b” produced an overflow. That’s exactly the sort of problem PBT is designed to detect: it checks minima and maxima, edge conditions, and so on, automatically.

I therefore restricted the range to @IntRange(max = 2147483622), which worked. That upper limit is the maximum integer minus the (integer) value of “z”, plus 1 because the minimum integer is one number further along than the max. I didn’t need to go that high for practical purposes, but the results told me just what I needed to know: the boundaries of where this code works and, probably more important, where it doesn’t.

By the way, a simple variation on the Caesar cipher is to start with a keyword, like Caesar, without duplicates (meaning CAESR) written under the letters ABCDE, then go with the rest of the alphabet, BDFGH…, skipping all the letters already in the keyword. The result is still a transposition cipher, with all the normal drawbacks (like the letter frequencies match those of regular English), but the result is a lot harder for a casual observer to break. Instead of a number, the shared key is a word, but still the same word is used for both encryption and decryption. Most of the history of code-breaking involves guessing (or stealing) a key like that shared by both sides.

In fact, shared-key cryptography was the heart of secret codes (and most spy stories) all the way up through the 1980s, when PKI (public key infrastructure) was developed. With PKI, each side has a public key and a private key, and messages encoded with one can be decoded with the other. That turns out to be the heart of all ecommerce on the internet, so it’s quite useful, but that’s a story for another time.

Star Trek: The Motion Picture

It’s really hard to communicate to people below a certain age how different popular culture was back in the 1960s and 70s. For those of us with a science fiction bent, in the 60s when Star Trek came along the impact was huge, not only because the biggest competitor was the silly Lost In Space show, but because for seemingly the first time somebody was taking the genre seriously.

Even so, the audience was sufficiently small that NBC cancelled the show after only two seasons. A huge letter-writing campaign got them to renew it for a third season, but the writing that year was not very good, and it got cancelled again. Of course I knew nothing about this at the time, partly because I was too young to understand and partly because we didn’t have an internet to tell us these things.

The show really caught on in syndication, however, running every night of the week continuously around dinner time for years. There were rumors it could come back for real, but all we got was Star Trek: The Animated Series in 1973 and 1974, which was (mostly) too silly even for fans like me.

Then in 1977 came the massive, totally unexpected success of a little independent film called Star Wars, and the world changed.

In 1979, Paramount released Star Trek: The Motion Picture, with all the original cast. I was thrilled, of course, at least until I watched the movie. It was lumbering and slow, and while most of the actors had aged well, some had not evolved the way I was hoping. Critics made fun of it (they referred to it as Star Trek: The Motionless Picture and Spockalypse Now) Still, the movie did well enough at the box office to start the franchise on its way again.

In late 1981 I was a sophomore at MIT when the Lecture Series Committee invited Gene Roddenberry to come to speak. He was a lot of fun, with stories about the production and the actors, but not a lot of information true fans didn’t already know. He did apologize for the first movie, saying they’d gotten carried away by the special effects, and promised the next one would be better. The next movie turned out to be Star Trek: The Wrath of Khan, which is still considered one of the best in the franchise history, so that worked out.

As another reminder of how much the world has changed, one of the women in the audience during his talk asked a question about Lieutenant Ilia, the Deltan who had to take a vow of celibacy in order to serve.

The questioner basically asked about the outright sexism that implied, and some related issues. I remember Roddenberry’s answer vividly. He shrugged and said, “That’s just how they are on that planet,” and the audience cracked up.

I know much more about what an extreme horn-dog Roddenberry was, even at the time (see the excellent Sex in Star Trek video by Jessie Gender for details). I’d like to believe that her question would be taken a lot more seriously today during the #metoo era, but there’s no way to know for sure.

The reason I’m bringing all this up is that two events happened involving the original movie:

A “director’s cut” was released back in 2001 that vastly improved the movie, as described in this article.
A remastered, 4K version of the movie came out this week on Paramount+. See this article in Variety for the original announcement. The new release is the director’s cut from 2001.

I watched it this week, because of course I did. I have to agree that the director’s cut (or “director’s edition,” as it’s called here) is much, much better than the original. Even though it’s only four minutes longer, they replaced a lot of the wandering around inside V’ger scenes with good scenes between the characters, and that helped a lot.

I’ll just summarize my impressions:

OMG, Spock looked fantastic in that meditation cloak:
I would totally wear that, though I have no chance of looking that good.
The planet Vulcan also looked amazing:
This was the first glimpse of Klingons with forehead ridges:
The Klingons spoke their own language, and so did the Vulcans. Despite what you may suspect, I never bought The Klingon Dictionary, nor did I learn the language. There were limits to by nerdom, and that was a step too far. As the saying goes, vIghIjbe'chugh vaj jIHbe' (I’m not that big of a nerd, in Klingon, at least according to this English to Klingon translation site).
My favorite quote from the movie came from Dr. McCoy, who looked like this when he made his initial appearance:
His quote was, speaking about changes to Sick Bay, “I know engineers. They love to change things.”
Spock was great, McCoy was great, Sulu, Scotty, Uhuru, and Chekov were all great, given what they had to do.
Kirk, however, was a huge disappointment. I don’t know who that guy with the poofy, wrong-colored hair was, but he wasn’t the calm, cool, powerful Hero of the Galaxy that that I admired as a kid.
Here he was insecure, manipulative, and practically begging for help (“Bones, I need you. Badly.”) It’s almost like he was a character on Star Trek: Discovery, where they routinely stop all the action just so the seriously-damaged characters can talk about their feelings. Ugh.

If you can get past that, however, it’s a pretty good movie. The director’s edition changes it from one of the worst to somewhere in the middle of the pack of the Star Trek movies, and it still looks gorgeous. The effects hold up very well, especially the “going to warp” effect that Star Wars shamed them into adding:

Also, the score by Jerry Goldsmith is wonderful, especially Ilia’s Theme, which I still love:

Beautiful.

Anyway, I recommend watching the movie if you get a chance. It’s way better than its reputation, and all the non-Kirk characters are great. Plus, in typical Roddenberry fashion:

As Harlan Ellison famously said about all Roddenberry stories, “The Enterprise meets God, and it’s a child, or a computer, or both.” Check on both counts.
The plot is ultimately resolved by a human being having sex with an omnipotent computer, because, after all, it’s a Roddenberry story.

What more could anybody want?

A Few Random Tweets

This is still going on, I think, during Week 1 of the Pragmatic Programmers Spring Sale:

PragmaticProgrammers @pragprog

Take concrete steps to build a relationship with your manager that helps both sides succeed. pragprog.com/titles/kkmanage 50% off with promo COMMUNICATION2022 #PragProgSpringSale @kenkousen

pragprog.com/titles/kkmanage/help-your-boss-help-you/

50% is a lot, so it’s worth checking to see if the promo code still works.

Spring Training

Ken Kousen @kenkousen

I know I'm easily amused, but I always enjoy doing Spring (Framework) training during Spring (baseball) training #Spring

I can’t believe I forgot to mention this during my Spring Training course during (MLB) Spring Training this week. MLB Opening day was Thursday, so that was my last chance to say it until next year. Oh well.

Early Wordle

Maria Scrivan @mariascrivan

#wordle

I’ll bet they always start with Bison.

Luck and Success

chessentials @Chessentials_

A really great video

youtu.beIs Success Luck or Hard Work?In a competitive world, tiny advantages can make all the difference. Get 10% off Snatoms with code ‘giveluck’ in the US: https://ve42.co/USA or International...

I say a lot about these issues in my Mental Bookmarks and the Fractal Nature of Success presentation on the NFJS tour, but I’ll leave that for another time. This video is very good, however.