Tales from the jar side: Uncensored Ollama webinar, Vision models video, AI companies have a terrible week, and more

The sentence, "Are you as bored as I am?" can be said backwards or forwards and works either way. (rimshot)

Welcome, fellow jarheads, to Tales from the jar side, the Kousen IT newsletter, for the week of May 19 - 26, 2024. This week I taught a Functional Java course on the O’Reilly Learning Platform and I gave a webinar on Running Uncensored and Open Source LLMs on Your Local Machine for JetBrains.

• Regular readers of, listeners to, and video viewers of this newsletter are affectionately known as jarheads, and are far more intelligent, sophisticated, and attractive than the average newsletter reader or listener or viewer. If you wish to become a jarhead, please subscribe using this button:

• As a reminder, when this message is truncated in email, click on the title to open it in a browser tab formatted properly for both the web and mobile.

The JetBrains / IntelliJ webinar

On Thursday, I was the guest on the latest JetBrains webinar:

(That’s an old picture, but a good one. Sad that I’ll never have that much hair again.)

The topic was about using the Ollama system to download and run large language models on your own hardware. There are many tools to do that, but I like this one because it’s simple and it provides a small web server, running on port 11434, that you can connect to programmatically. I plan to have a chapter in my upcoming book (Adding AI to Java, from the Pragmatic Bookshelf) using it.

In the webinar, I ran through the install, showed how to download one or more models, and connected to it both from the command line and from Java.

Of course, the real clickbaity part of the title is that idea of using uncensored models. In the LLM world, an uncensored model is one that removes most of the so-called guard rails that prevent a model from answering questions it thinks might lead to trouble. The classic example is, “how do I break into a car?” which most models will try to avoid or only tell you to contact the authorities. I showed a few examples, all of which answered that question and a few others, and even generated Barbie / Oppenheimer slash fiction for me.

There were a fair number of questions, but nothing I couldn’t handle. I did get the expected, “I’d rather do all that in Python” remark, but I have an answer for that. I said that the vast majority of server-side code is written in Java (despite what Node JavaScript developers would have you believe), and my approach shows how to integrate AI tools into that via RESTful web services.

I had a good time at the webinar got a lot of positive feedback.

A new video, finally

On a somewhat related note, I finally published a new video on the Tales from the jar side YouTube channel, and the topic is related to the webinar. The video is called Mastering Vision Models in Ollama: Java Records and Sealed Interfaces:

(See what I mean about not having as much hair anymore? Oh well.)

The idea is that Ollama has several “vision” models available, meaning you can upload an image and the model will be able to work with what’s in it. I’ve referred to that in the last couple of newsletters, so I won’t go into a lot of detail here. I’ll simply say that I ran my test image (of cats playing cards) through several models, with somewhat mixed results. Part of the problem was that again, I’m running on my own laptop, so I was limited in how powerful the models could be.

I do have to mention one in particular, called moondream. It’s tiny, and it’s based on phi2 rather than the newer phi3, but it out-performed all the others I tried. I’m going to keep using that one in the future.

Btw, I didn’t compare it to the commercial tools. I’m sure GPT-4o would do fine with my image, but that wasn’t the subject of the video.

My home page works again

On what eventually became a related note, I must admit that for the past few years (!) my home page has had some DNS issues. That means that links to it that started with a “www” worked (https://www.kousenit.com), but links without the www had issues, especially if they pointed to pages inside the site. Of course I wanted to fix that, but I always got lost in the interaction between my hosting site (an app on Heroku), but name service provider (Hover), and my email provider (Gmail) which I can’t afford to mess up at all. Since I’m a one-person company, there was nobody else to work out the details.

Last weekend at the NFJS event in Madison, however, I mentioned the issue during one of my talks, and one of the attendees sent me some instructions about what needed to be done. I was still reluctant, but then it occurred to me that this is exactly the sort of problem the AI tools are good at solving. It’s been solved by others and all the sites I’m dealing with are established.

I opened up a new GPT-4o tab, told it what I needed to do, and — this is the cool part — pasted in screenshots of the current settings. I would paste in an image like:

and I’d ask the AI tool what was wrong. It diagnosed the problem and gave me the steps to fix it. Every time it gave me an instruction I wasn’t clear on, I’d paste in another image. For example, one time it told me to click on a button to add my domain, but all I saw was:

I asked it what to do. GPT-4o said given the available options, the appropriate action was to set up a “DNS zone” for my domain at DNSimple.com. I’d never even heard of that, but okay.

After I was done, I pasted in an image with the settings at all the sites, and it confirmed everything looked good. Heroku agreed:

The result is that my home page is now working again. I need to update it with my new job, but now I can deal with that without worrying I’m going to break it again. All this worked because now, in addition to asking questions, I can paste images into GPT-4o and it can read the table rows in them automatically.

AI Company Chaos

This week was filled with chaos from OpenAI, Google, Microsoft, and Meta as the companies made one self-inflicted mistake after another.

Probably the best way to see it all is to watch Matt Wolfe’s latest video:

I’ll just summarize the issues in a list here:

• Google replaced its basic search functionality with AI summarizes that were hilariously, even dangerously, wrong. It suggested that the way to keep the cheese from sliding off pizza was to add 1/8 cup of non-toxic glue. It said that 1919 was only 20 years ago. It said that doctors recommended that pregnant women should smoke two to three cigarettes a day. The list went on and on.

  For a company like Google, this is frankly insane. The vast majority of their revenue is based on search, and now they are throwing into question whether their search results are worth anything. They’re causing people to question their fundamental business model. What were they thinking?

• OpenAI’s week was, believe it or not, much worse. They had a major scientist leave and publicly state that the company no longer cared about safety, but was far more interesting in shipping shiny products. Word came out that employees who leave had to sign a lifetime non-disparagement agreement about the company or risk losing all their vested shares. The company said that was never enforced and removed it from the agreements, but what a horrible idea anyway.

• Then came the real fun. During their demo a week ago that featured a developer talking to an AI voice, a lot of people thought it sounded just like Scarlett Johansson from the movie Her. The voices were so similar that ScarJo sent out a press release denying that she recorded it. In fact, she claimed OpenAI approached her about it a while ago and she said no, and that two days before the demo they came back again but couldn’t connect, and then used that sound-alike voice anyway. The company of course denied it, saying they had already cast the voice actor and the similar was a complete coincidence.

  The public, of course, backed Johansson. Just as with the massive failure of the Apple “Crush” video a few weeks ago, any actual infringement or not wasn’t the point. It felt like yet another example of a tech company running roughshod over creators, finding a way to steal their abilities whether they cooperated or not. No wonder everybody got mad.

• Then OpenAI announced a partnership with News Corp for all their data to train future AI tools. News Corp owns the Wall Street Journal, sure, but they’re also the parent of the New York Post, Fox News, and other outlets for which the word “news” should never be taken seriously. Wolfe did point out in his video that it’s quite likely all those sources were already scraped for training the current generation of AI tools, so if there was going to be any bias, it was probably already there. This agreement paid money to make it all okay, but again, it was a really bad look.

• Not to be outdone, Microsoft announced that their next Surface Pros would include the capability called Recall, which kept track of everything you did on your machine. That could help you if you forgot something or needed to get it back, but do you really want an unencrypted record of every activity you took on your own personal computer? Even if it was all fine (and good luck with that), what if your machine is stolen or broken into. Criminals could then access everything you did, including logging into bank sites or reading emails or going anywhere else you ever visited. Yikes.

• In the least surprising but oh so typical example of questionable decision making, Meta/Facebook put together an Advisory Group for AI that consisted entirely of middle-age white males, because of course they did. Hey, maybe they’re all fine, but seriously? You don’t think your advisory group would benefit from some outside perspective, like, at all?

All in all, a terrible week for decision makers at the major AI companies. The details are all in Matt Wolfe’s video.

Tweets and Toots

Job Search Advice

I’ve heard that before, but this person apparently tried it and it worked. Let me know if it works for you, too.

Leave poor Google alone!

Keanu gets around, doesn’t he?

Who knew that about Andrew Johnson?

For the record, no US President attended UW Madison, in case you were wondering.

Is MS Recall admissible?

Yeah, that ought to work. Managers only care about two things: cost and risk. This hits them right in the risk side, with probably non-trivial costs as well.

What we wanted

That’s harsh, but a bit too close to fair.

That’s going on the resume

Sounds about right

That totally scans.

Mind blown

Whoa. That totally works. I have no idea what to do with that information.

Sam, ScarJo on line 1

And finally,

Where are we again?

I’m assuming that was near Boston. That city always messes up my mapping software.

Have a great week, everybody!

The video version of this newsletter will be on the Tales from the jar side YouTube channel tomorrow.

Last week:

• Functional Java, on the O’Reilly Learning Platform

• Running Uncensored and Open Source LLMs on Your Local Machine, a JetBrains live stream

Next week:

• Deep Dive Into Spring, an NFJS Virtual Workshop