Blackcap Blog

Published at 09:56 on 30 July 2021

Trees Are Stupid

Not the living beings (those are amazing), the data structures. The things my undergrad CS teachers were obsessed with assigning tedious programming exercises to implement.

I am reviewing how to pass those stupid coding tests most interviewers seem to be so fond of these days, and one if the things that has become clear is just why I never much liked trees in the first place. In short, if you use trees, you are virtually always stuck with two choices: a simple, logical, easy-to-understand tree that is vulnerable to pathological behavior, or a complex, quirky, difficult-to-understand one that literally makes buggy code inevitable.

Yet my professors pissed away so much time blathering about trees and how to code them. How many times have I used, I mean actually used, such knowledge professionally? Maybe once or twice in my decades-long career. Not surprising, given their many disadvantages.

Consider associative arrays (a.k.a. dictionaries or hash maps), which all modern programming languages support to some degree. These are versatile general-purpose data structures that eliminate much low-level grunt work. A huge part of the reason I so seldom use trees is that I just use associative arrays instead.

Ah, but don’t these use trees under the hood? No, generally they do not. They use hashing functions, arrays, and linked lists. Why? Because you typically get faster access that way, and they are simpler to code (and therefore have fewer chances of implementation bugs), that’s why. The people who code language runtimes and standard libraries are not stupid.

Yes, even in what is alleged to be the canonical “trees excel at this” case, they actually don’t. Not really.

Sure, trees have some genuine uses in databases. But here’s the thing about databases: very few people write them. This is because few people need to write them. The rest of us simply use them, and there is already a very nice set of databases (well debugged) out there ready to be used. Why re-invent the wheel, particularly when it would involve so much tedious, bug-prone coding?

So it’s not that trees are useless, it’s just that they are far less useful than their prominence in the undergraduate computer science curriculum would indicate.

Why are they so common, then? One must consider the general purpose of higher education: to furnish to the economic system individuals that are screened and graded for both intelligence and obedience to authority. Seeing who is motivated to perform meaningless, tedious programming exercises is a great way to do this. As a bonus, many of the more sophisticated tree structures have the advantage of requiring algorithms that are both non-trivial enough to give students a good coding exercise yet trivial enough for them to be expected to code in the first place, thus making them a good source of busywork.

Job Interview Coding Tests Are Stupid

They are stupid precisely because they inevitably want you to show off your tree-coding knowledge. But as we have seen, in the real world, you don’t directly touch trees. You use a database or a hash map, and be done with it. This goes for a surprising amount of the generalized stuff they teach you in college, in fact.

Seeing the world in terms of general-purpose principles can often be positively harmful in programming. Some of the most useful code I have written for people is code that was proclaimed by others to be impossible to write, because it was an instance of a generally impossible problem to efficiently solve. While this was true in the general case, a little introspection into the particular instance of the problem at hand would reveal it had specific attributes that made a solution possible. The general case would break my code, but that was irrelevant because my code was not running against the general case. A solution was possible.

In one case, there was so much resistance from others in the team that I had to sneak my code into the system. It was only some weeks later that I observed how things were “mysteriously” behaving better, and then pointed out the reason to my incredulous teammates, who were at that point compelled to concede that the problem was solvable after all.

Success in real-world coding is based on being able to determine the unique and specific characteristics of a particular problem, how these differ from the more general cases of problems, and how to leverage these particulars to craft specific solutions that are significantly more efficient than any stock algorithms or data structures could ever be.

I will make an exception here for the coding test that my passing led me to get hired at the best job I have ever worked at (at least for the initial several years, until both the job and the company changed to the point that I was no longer well-suited to the position). It was crafted to have such realistic special characteristics, and it was my quick spotting of these that impressed my interviewers.

But that was the exception that proved the general rule: interview coding tests are stupid.

Published at 11:16 on 6 July 2021

Is the process of cutover to my new hosting solution (i.e. self-hosted) done? We shall see.

One wrinkle is that my self-hosted email server seems to be DNS blackholed. Hopefully I can resolve that. This is a virtual host, and the IP address it possesses may have been used by an incautious or abusive site in the past. Unfortunately, it is not possible for me to preserve my old, known-reputable IP address. This is yet another instance of a problem where abusive Internet users cause headaches for the vast majority of non-abusive users.

Update. Almost done, it turns out. The emails from the new server are being rejected by both Apple and Google, because my new static IP address is for some reason on a blacklist. Guilty until proven innocent, oh joy. Now I must argue to have my address un-blacklisted. Mostly I blame spammers and not Apple or Google; I have used such blacklists myself in the past and may well do so again in the future. Abusers of the Internet have ruined so much of it for honest users.

Published at 19:03 on 19 June 2021

I might (finally) be (almost) done with this blog upgrade. We shall see.

I will say that WordPress does not make it easy to move a blog to a new server. There is a defined processs for purportedly doing this. Pity it does not work very well:

1. It has a rather silly 2 megabyte limit, one for which the recommended process does not remove. I eventually found a post from another user who had fought the same battle that explained how to do it, but wait, there’s more.
2. The pathetic P.O.S. does not preserve permalinks! Congratulations, every internal link to another blog post is now broken.
3. For some reason, it gratuitously replaces paragraph breaks with line breaks. WTF?

This is more than a little bit disappointing, because one would expect WordPress to be able to properly import from another site running the exact same software. I’m lucky, because I’m a computer geek. My solution was to go into the MySQL command prompt and do some database surgery:

1. Make sure the two blogs are running the exact same version of WordPress.
2. Drop all existing tables.
3. Restore from a database dump taken on the old system.
4. Go into the wp_options table and set siteurl and home to reflect the blog URL on the new host.

The average user, however, would be S.O.L. Most people don’t even know what SQL is, let alone know enough of both it and general database design principles to be able to engage in the sort of hackery I just did. They would be stuck not being able to move their blog, stuck with a broken, damaged new blog, or stuck with the lengthy and painful job of repairing their damaged new blog completely by hand.

Published at 17:56 on 15 June 2021

This is probably going to be part of a series about the curve balls Ubuntu 20.04 LTS throws at the veteran Ubuntu Linux user.

When you install MySQL with:

apt-get install mysql-client mysql-server

You will get an oddly-configured MySQL server that uses a newfangled thing called auth_socket authentication for the root user. The upshot is that you will not be able to log in to mysql as root unless you are already the root Linux user, and in the latter case you will always be able to log in, regardless of what password you supply, or even if you supply a password at all.

If, like me, you are logged in as the root Linux user (and why wouldn’t you be, if you are doing a system install), then it appears as if authentication is completely disabled, and your mysql server’s root account is wide-open. At that point, you will try doing Internet searches to uncover the cause of the problem, and if you are like me, you will spend hours trying different keyword variations and finding exactly nothing pertinent.

The fix is to change the root user to use caching_sha2_password authentication and set a password for it, e.g.:

ALTER USER 'root'@'localhost' IDENTIFIED WITH 'caching_sha2_password' BY 'iSpQ7U9c8kGz';
FLUSH PRIVILEGES;

(And no, that is not my actual root password.)

Published at 08:22 on 19 April 2021

Executive summary: It’s not that easy, but it’s not really all that hard, either.

The main complication is, well, complication. WordPress is a mature and very full-featured software system, and web layout itself is intrinsically a not-so-simple thing, because the same layout must work well on both a big desktop screen and a tiny smartphone screen.

If you start by editing the existing WordPress default theme, you will quickly get lost in the complexity. It is far better to start with a simplified, bare-bones theme like Tania Rascia’s untheme and work from there. The rub is, such bare-bones themes tend to be too simplified, so you will then be compelled to add the missing features you desire.

This still is working out quite well for me, because I am getting a theme that has the features I want and none of the features I do not want, resulting in a very clean layout overall.

The design itself is based loosely on mid-twentieth century newspaper typography. The typefaces come pretty close, and I justify my body text margins, but column width and paragraph style follows more typical Web standards.

This is yet another stage in the process of moving this blog off of its current host and onto the virtual host where the rest of my online presence is kept.

Published at 13:51 on 17 April 2021

Wanting to plan for the coming day where this blog moves to a new host, and wanting to try alternatives to PHP-based blogging (like WordPress) I decided to create a test blog using Roller.

The first thing that stuck we was how many configuration steps there were. Then I noticed how needlessly repetitive the configuration was; some things, like the database connection URL, must be specified in multiple places. That’s horrible. It’s literally begging for inconsistencies to crop up later. Configuration parameters should only need to be specified once. Then I run into ambiguities in the documentation; locations for files I am being told to create are not explicitly specified anywhere.

Finally I get things configured, only to receive a curt message from Tomcat that it cannot run my Roller instance because the Java was compiled against a JVM that is newer than the Java 8 JVM that runs my existing Tomcat instance.

I could fix that, of course, but f*ck it. Nowhere in the release notes was this dependency specified, and at this point I’ve already pissed away well over an hour and have yet to see so much as a blank dummy page for my efforts.

I notice that I already have PHP installed anyhow, because I needed it to run another canned solution at one time. I shelve the effort and give WordPress a try. Within fifteen minutes of following the install instructions (which are complete), I am rewarded with my blank dummy page. No multiple configuration files that repeat the same parameters. No ambiguous instructions. No unstated dependencies on PHP versions (there are dependencies, but they are clearly stated). It just works, as advertised.

Take it away, Coding Horror:

I’ve written both VB and PHP code, and in my opinion the comparison is grossly unfair to Visual Basic. Does PHP suck? Of course it sucks. Did you read any of the links in Tim’s blog entry? It’s a galactic supernova of incomprehensibly colossal, mind–bendingly awful suck. If you sit down to program in PHP and have even an ounce of programming talent in your entire body, there’s no possible way to draw any other conclusion. It’s inescapable.

But I’m also here to tell you that doesn’t matter.

The TIOBE community index I linked above? It’s written in PHP. Wikipedia, which is likely to be on the first page of anything you search for these days? Written in PHP. Digg, the social bookmarking service so wildly popular that a front page link can crush the beefiest of webservers? Written in PHP. WordPress, arguably the most popular blogging solution available at the moment? Written in PHP. YouTube, the most widely known video sharing site on the internet? Written in PHP. Facebook, the current billion-dollar zombie-poking social networking darling of venture capitalists everywhere? Written in PHP. (Update: While YouTube was originally written in PHP, it migrated to Python fairly early on, per Matt Cutts and Guido van Rossum.)

And the best thing about it? Although WordPress is written in PHP, it’s basically written (past tense) in PHP. I don’t have to do much with PHP’s awfulness myself. Edit some manifest constants in wp-config.php, and (since I am tinkering with templates) edit some .php files that are mostly parameterized HTML with a few PHP function calls. And what there is, is thoroughly documented, because WordPress is used so much.

Sure, it sucks that WordPress is written in PHP. But, it doesn’t matter.

Published at 13:27 on 6 April 2021

I am in the process of developing and publishing an Android app to the Google Play store. Part of the process of doing so is developing and publishing a privacy policy.

Initially, I thought this would be super-simple: Don’t collect information, then there is nothing to share or to establish policies about sharing. Simple. However, in the real world, things are seldom so simple as they might at first appear.

The first complication came when I realized that although my app does not (and probably never will) gather and pass on usage statistics, the places from which users might download my app, which will include a web site run by yours truly in addition to the Google Play store, certainly will gather such statistics.

Virtually every web server on the Internet logs each and every request it receives, and these log messages typically contain, at a bare minimum:

• The time a request arrived.
• The IP address the request arrived from,
• The URL of the resource being requested, and
• Basic information on the user agent (i.e. web browser) used to make the request. Such information typically includes the operating system that the user agent was running under.

So, say you are an AT&T customer in Brooklyn who uses your Samsung Galaxy S21 to download a copy of my app. I (or Google) will be able to tell from your IP address that you are an AT&T customer in the New York City metro area. We may even be able to tell that you were in the borough of Brooklyn, and that you were using a Galaxy S21. If we share your IP address with AT&T Wireless, they will be definitely able to determine exactly who you are, what hardware you used, where you used it, and (if you were doing something unlawful and/or abusive) take action against you for what you did.

Some Internet users are shocked to discover this. If you are one of those, consider yourself educated.

Why is this done? Not always for nefarious purposes! In fact, not usually for such. Gathering such data can be extremely useful for dealing with things like abusive users (they exist), troubleshooting software and network problems (they are inevitable), or managing the growth of traffic to a web site or to a cellular network.

But it’s still pretty simple, right? So I am collecting basic usage statistics (and Google Play will doubtless collect some on my behalf that it can share with me in reports). Just do not share the information!

Well, there is the matter that I could end up in jail on a contempt of court charge for adhering to such a policy: what if a law enforcement officer or a process server arrives at my door armed with a warrant or a subpoena?

Okay, then, exclude that and nothing else. Solved!

Not so fast, yet again! What if my app becomes popular with violent white nationalists and neofascists? I am, after all, promising to gather a fairly minimum amount of information and to be as reluctant as possible in sharing it; that makes my app attractive to such individuals.

It also makes it attractive to those breaking laws to undermine oppression and to advocate for more freedom, which is my main intent. If that sounds reckless to you, just ponder that any oppressive order has always considered it a crime to undermine said order; revolutionary politics is intrinsically criminal politics. Lech Wałęsa was a criminal; Martin Luther King was a criminal; Mahatma Gandhi was a criminal. If the Founding Fathers of the United States had failed in their endeavor, they would have been prosecuted and for the most part executed for the crime of treason against the British Empire.

The only exceptions to the above rule are certain situations when the revolutionaries are judged to be sufficiently tiny in number and powerless so as to pose little or no threat to the established order. And as soon as they gain enough power to cease being so, watch out! The velvet gloves will be replaced by an iron fist.

But I digress. So now I must craft an exception for things like neofascist and white nationalist politics. While I do not want to, and do not have any intent to, regularly monitor the download logs, I want to be free to cooperate with antifascist organizations should my cooperation prove helpful to the cause of fighting fascism.

That, of course, begs the question of just what, precisely “neofascist and white nationalist politics” is. However I define it, it opens up the prospects of all sorts of word games: “No, I am not a ‘fascist,’ you stupid leftist. I am a ‘nationalist’ and an ‘identitarian.’”

Now I am stuck trying to anticipate those word games, all the while also having a privacy promise that still is meaningful to the vast majority of people, even people whom I might politically disagree with, who are nonetheless not fascists and whose beliefs must be accepted as part of the diverse spectrum of beliefs in any free and open society.

In the real world, things are seldom so simple as they might at first appear.

Published at 09:36 on 3 April 2021

It leaves a lot to be desired.

The normal unit testing is advertised as supporting most of the Android class library (which is not the same as the standard Java class library), but what they don’t tell you is that it’s chock full of stub-out dummy logic. The routine to load an image from a file, for example, always returns a 100 by 100 black image. That’s sort of a deal-killer if one is trying to test image-processing code.

The instrumented testing runs on Android devices so avoids those headaches, but it too is extremely limited in scope and needlessly developer-hostile. For example, the test code is by default strictly disallowed from making any modifications to the filesystem. If one is testing an app that processes files, that again ends up being a deal-killer (how, exactly, am I to create the test files to feed to the app being tested)?

There are ways to disable this misfeature, but they are very poorly documented. It’s a setting buried deeply in an obscure settings menu somewhere. Where, exactly, is not standardized: it varies from device to device so much that one set of instructions is not even valid for a single Android OS release. I gave up in disgust after pissing away at least an hour searching in vain for it on my phone.

If Google wants developers to write good, comprehensive tests for apps, they need to stop making it as difficult as possible for us to do so. Until then, Google can take its pleading about writing tests and go fuck themselves. I will still write tests, but not very comprehensive ones.

Published at 15:26 on 31 March 2021

The world is full of analyses like this one that confidently perform crimping to be better than soldering. The real world is not nearly so simple.

Yes, a properly executed crimp connection with a quality crimp connector is by all measures superior. The devil is in those weasel words.

Given that it is possible for a crimped connection to be superior to a soldered one, and given that crimping is faster than soldering, why would anyone solder? Soldering when connections can be crimped seems obsolete.

That is how many retail hardware stores promote crimping, often in a big blister pack with cheap crimp connectors and a cheap crimping tool like this one. Well, good luck with that. It takes a skilled craftsman to execute a quality crimp with a cheapo tool and cheapo connectors. It is, in fact, easier to learn to solder.

An anecdote to close: When I worked in IT support, the department purchased a cheap crimping tool, that could crimp both 6 and 8-position modular connectors, and some bulk cable. No longer would custom lengths of cable need to be special ordered.

Those crimps were responsible for trouble ticket after trouble ticket. When I broke the crimpers in the attempt to exert enough force for a quality crimp, I put my foot down and insisted they spend over $100 on a name-brand, quality crimping tool and set of crimping dies. It was money well spent, because the number of trouble tickets dropped to zero on connectors crimped with it.

It’s not that bad with standard wire crimp connectors; $25 or so can get you a good, compound-action, ratchet-based crimping tool. Even then, it’s good to budget in some practicing, and learning how to recognize a bad crimp. But again, that’s not how crimping is sold. Most of those crimp kits don’t even cost $25 total, and no mention is made of skill development.

Personally, I solder. Already have a soldering iron and know how to use it as a result of messing with electronics for many years, and I don’t splice wires often enough to justify the expense of a crimping tools, the clutter managment headaches of maintaining a stock of crimp connectors, and so on.

Published at 07:58 on 28 July 2020

Say you’re doing some coding and debugging using the command line. You change directories to where the source code is and thenceforth, until you explicitly change to some other directory, you have set the default directory for all editors, compilers, debuggers, file-filtering utilities, and so on. It’s automatically understood that, unless you explicitly specify otherwise, if you specify a file name, it will be in the current working directory.

No graphical user interface that I’ve used has had anything analogous. As such, I’m forced to continually spend effort manually specifying source and destination directories. This constitutes repeated work that I don’t have to do when using the command line.

Each utility starts out relative to the home directory, or, in some cases, the directory where it was last used. It’s hard to say which is worse, but I think my vote goes for the latter: it’s a pathetic attempt at user-friendliness that ends up being incredibly user hostile: in order to predict where the files for a given application will go, I must memorize, for each application, where the last file I specified with it lived.

It all results in continual violations of the principle of least surprise. I’m persistently having to use find to determine which seemingly random and bizarre place a GUI application just created an output file in.

This, more than anything else, is why, decades into the existence of the graphical user interface, I inevitably have one or more terminal windows open, and opt to do a significant amount of my work at the command line.