Blackcap Blog

Published at 12:02 on 31 May 2024

Foreword

I have thought of writing this article more than once before. I just got assigned yet another HackerRank test as part of an interview process. Sometimes, I have blown such things off entirely. Sometimes, I attempt to do my best on answering them.

This time, I plan to do something different. I will have a link here in my response to the problem; if you are reading this as a result of following that link, then here is why I responded the way I did.

Introduction

HackerRank sucks for two main reasons:

1. The general irrelevance of undergraduate computer science exercises to real-world programming, and
2. The artificiality of HackerRank’s time constraints.

Irrelevance of Problems

In my experience, HackerRank’s exercises tend to be thinly-disguised rehashings of undergraduate computer science homework assignments. Such assignments tend to be heavily based on coding implementations of basic data structures, often trees of various sorts, that have very limited real-world relevance to solving problems on the job. I have written on this here before. It’s really not a surprise, given how the usage of such problems is basically mandated by the general nature of the constraints of the HackerRank platform.

Effectively, such tests screen for either recency of undergraduate coursework (and thus lack of practical, on-the-job experience), or willingness to spend time brushing up on a skill set whose real-world utility is extremely limited (and thus for unquestioning submission to authority and willingness to obey pointless rules). The first is diametrically opposed to the goal of filling a senior-level position, and the second is diametrically opposed to the sort of environment that I personally thrive in.

Artificiality of Time Constraints

Real-world programming tasks generally do not pop unexpectedly out of nowhere, with no advance warning. They are usually foreseeable as part of the natural trends of evolution of software systems and the organizations they support. As such, the prudent developer has usually already spent some time thinking about such issues. Sometimes, of course, they do pop up unannounced (long-latent bugs sometimes manifest, and sometimes have severe impact). Even then, they don’t come with a count-down timer and a hard artificial deadline. It is possible to take a walk (I get some of my best ideas outdoors) or to bounce ideas off colleagues.

With HackerRank, there is no such subtlety. The clock is ticking, the artificial deadline is rapidly approaching, and it will be enforced without mercy. (Sometimes you even get a so-called proctored test, in which you must enable a spy camera. Leaving the room is considered cheating. Lucky you.) You will solve the problem in one sitting, and you will do it now. And the problem has limited real-world relevance at best.

Conclusion

Maybe I will submit an actual solution, and maybe I will not. It will depend on the exercise and how I feel about it at the time I submit my response. At any rate, the link here will be the most important part of that response.

Really, there is not much more to say, so it is time to wrap this up. HackerRank sucks, and now you know why.

Published at 15:01 on 11 April 2024

After a recent update, my iPhone started letting me control my headphone volume from threshold of pain loud to insane instant deafness loud. I guess some aging hipster who ruined his hearing going to too many rock concerts without hearing protection got appointed to a QC position at Apple.

Based on what I had read about human sound perception, I guessed I needed about 6 dB of attenuation to tame the thing. A simple matter of adding four resistors to the picture (two for each channel, one in series to cut the voltage in half, and another in parallel to restore the impedance the audio amplifier sees to what used to be pre-attenuator).

The worst part about it was all the fiddly soldering (those connectors have some tiny terminals). But it works, and 6 dB was indeed the correct amount of attenuation needed to restore sanity to the device.

Published at 23:31 on 2 April 2024

It’s the math.

Specifically, they all (at least all the leading ones: Microsoft Excel, Apple Numbers, and Libre Office Calc) use floating point numbers and arithmetic.

Fractional radix digits are only capable of accurately representing numbers whose prime factors contain only factors of the radix. For the sort of base 10 numbers we are familiar with, this means that any fraction whose denominator can be represented as a product of 2’s and 5’s can be represented. As an example, 8 factors to 2³, so eighths can be represented with complete accuracy as decimal fractions. It takes three digits, of course, because 10³ (2³ ✕ 5³) is the lowest power of 10 that is an even multiple of 8, but you can do it. And really, three digits isn’t that bad.

If you use a denominator that cannot so be represented, then you get an infinitely-long repeating fractional part. The canonical example of this is ⅓ turning into 0.3333333….

But computers use base 2, not base 10, and this creates a problem. 2 is itself prime, so fractional radix digits in binary notation can only represent denominators that are powers of 2 accurately, and nothing else. Everything else turns into a number with an infinitely-long repeating fractional part.

This is a big problem, because one of the most common uses of spreadsheets is financial calculations, and floating point number can only represent monetary amounts as small as 25 cents accurately. If a financial quantity does not end in .00, .25, .50, or .75, your spreadsheet is representing it wrong! Only slightly wrong, of course, but still wrong. And if you are adding and subtracting enough numbers together, eventually the result will be wrong by a penny or two.

It is for this reasons that banks use decimal arithmetic, not a processor’s built-in floating point arithmetic, for their financial calculations. They don’t want their customers’ balances to drift from reality by a few pennies per year. Banks have done this since just about forever. COBOL, one of the oldest high-level programming languages out there, and designed for business computing, uses decimal arithmetic by default, and this is why.

The rationale for spreadsheets not doing likewise is for “performance” reasons, but frankly, that is a load of horse hockey. Yes, built-in floating point calculations are faster. But the performance hit from using decimal arithmetic is far from a deal-killer. COBOL dates from around 1960, when computers had only a tiny fraction of the computing power they do today, yet COBOL programs ran just fine way back then, and cranked out accurate results without gratuitous rounding errors. (Plus, your average spreadsheet is a lot smaller than your average batch of bank transactions to process.)

I was going to make more use of spreadsheets in figuring my income taxes this year, but after learning the above I am mostly sticking with good old dc, which uses decimal arithmetic. (Actually, it uses base 100, but when it comes to avoiding rounding errors, base 100 works identically to base 10, since the latter is a power of the former.)

Published at 22:44 on 7 February 2024

It’s the libraries, stupid.

The standard Swift library is laughably in-comprehensive. Things you can do in the standard libraries for Java, Python, PHP, C#, Ruby, and most other common modern programming languages just aren’t in there.

What you are supposed to do, from what I gather, is to use the Apple Foundation framework. There are several problems with that:

• The framework is a hot mess. It got its start back in the 1990’s as part of the NeXT operating system, and has been incrementally hacked on ever since. The documentation is likewise a mess: incomplete, cryptic, and poorly-organized. It is fully part of the pattern that Apple products tend to be as programmer-hostile as they are user-friendly.
• The framework is still incomplete. Support for tasks as basic as doing buffered reads from an arbitrary text file on a line-by-line basis are absent from it. (At least I think they are absent; review the part about the documentation being a hot mess above.)
• The framework is an Apple-only thing. There is an ongoing effort to open-source the Foundation framework so that Swift programs can be more portable, but it is a work in progress.

The bottom line is that Swift is not, in fact, the general-purpose programming language it is claimed to be. Unless one is writing native-mode GUI applications for Apple products, Swift really doesn’t make much sense.

It’s a shame, as the core Swift language looks to be fairly well-designed. It could be a great general-purpose programming language if only it came with a decent standard library. Alas, that’s a bit like saying Mr. and Mrs. Lincoln could have had an enjoyable evening at Ford’s Theatre if only that hadn’t happened.

I may eventually delve into Swift for such purposes, but as things currently stand, programming in Kotlin with the Java Swing platform allows me to develop GUI tools that run on my Mac, and I don’t have to deal with all the ugliness that is Apple’s native programming environment. Swing isn’t perfect, and its rough edges sometimes manifest, but it’s been good enough for my personal use.

Published at 23:22 on 27 October 2023

It is a known bug. The workaround is to use fontspec to invoke the feature manually, e.g.:

\fontspec{Baskerville}[Renderer=OpenType, RawFeature={+smcp;-liga}]

Ligature substitution should be disabled when using small caps because the two features tend to be incompatible.

Published at 18:03 on 22 October 2023

Python is already sitting at the top of the TIOBE Index of most popular programming languages, and has been for some time. And no wonder: it’s one of the best ones out there.

One big thing that stops it from being close to the best is that it has difficulty walking and chewing gum at the same time. In one project, this caused me no small amount of pain. It’s part of the reason I have used the Java virtual machine (usually via Kotlin, which is a more modern language than Java) on some of my projects.

Over the years, there have been numerous proposals to remove the global interpreter lock (GIL) from Python. These have generally gone nowhere, with the exception of the existence of Python versions that target the Java virtual machine and the .NET common language runtime*. There are a number of valid reasons for this.

But now, there is a very serious proposal to remove the GIL from the reference implementation, and the Python Steering Committee has indicated they will almost certainly accept it.

Once this happens, expect Python’s dominance to increase further.

* Alas, these implementations tend to lag (sometimes seriously) behind the reference implementation, plus they are not compatible with many of the third-party Python libraries out there. (The latter issue is also why it has been so difficult to remove the GIL, as doing so in ways that are both a) not ruinously inefficient and b) compatible with existing libraries has proven exceptionally difficult.)

Published at 15:27 on 21 October 2023

I held considerable hope for XeTeX (the modern-day TeX). Alas, while arguably better than Groff, it still leaves a lot to be desired in the font department.

Namely, while it can indeed load and use the same standard system fonts that all other programs can (a big win over classic TeX), its support for OpenType font features is quite limited and lacking. For instance, small capitals don’t work. I have tried to use them multiple ways, including directly via RawFeature=+smcp, and either nothing happens and I get normal mixed case, or I get a complaint that Font shape `TU/Baskerville(0)/m/sc' undefined and again I get normal mixed case.

I know the smcp (i.e. set lowercase input in small caps) feature is present in the font I am using, because I can use it from Libre Office. Apparently, XeTeX just doesn’t get enough use for this sort of thing to get adequately exercised. At this stage, I’m getting to the point of writing the TeX family of text formatters off as hopelessly yesteryear.

The feature works in some other fonts I have installed, so apparently it’s a bug that only affects certain cases. Unfortunately, one of those cases is in the font I most wish to use for this project.

And yes, I know I could extract the small caps myself and create an .otf file whose lowercase is small caps, and load that font. Well, eff that. Not having to do such awkward hackery was my whole motive for installing XeTeX in the first place. If you say your tool supports standard font files, it should support the standard features in those files directly.

It’s a disappointment, as the well-documented plain-text input of formatters like Groff and LaTeX make them quite useful for formatting automatically-generated output. Modern font support has long been, and apparently continues to be, their Achilles’ heel.

Published at 00:22 on 14 October 2023

Probably the textbook example of why Java sucks is the Log4Shell Bug in the log4j library.

It’s just a library and not part of the core Java language, but the core Java language really isn’t the problem. The problem is the dysfunctional culture that surrounds the language.

So the stock logging is both bloated and yet surprisingly limited in functionality, because that is the sort of code the Java community tends to produce. That creates the need for an expanded logging system. Since it fills a void, everyone uses it.

Since the community does not appreciate simplicity, what the hey, let’s stick a general-purpose templating language into the thing. The same community wrote that general-purpose templating language, so what the hey, let’s stick a shell escape with command-output substitution into the templating language. Both the general-purpose templating and the shell-escape feature of the templating are of course enabled by default, because what the hey, why not?

And suddenly, we have a logging library with a shell escape in it.

Just how bad this all is, is underscored by how it sat unnoticed for eight years before it started being exploited. That’s right, so few people actually used all this creeping feature-ism that it took nearly a decade for the vulnerability in it to be discovered!

Look, logging and templating are two different domains. Templating and launching commands in a subprocess are two different domains. A logging package has no business containing templating beyond approximately the String.format level. A templating package has no business containing general-purpose subprocess creation.

If you really have a need to generate a super-long, super-complex log message, long enough and complex enough to require a general-purpose templating system, you should probably rethink what you are doing. Log messages should be relatively short. Now, maybe your case is 1-in-1,000 to 1-in-1,000,000 special and there really is a good excuse for a long message. Maybe. Fine. Do it by hand. Pull in a templating library, use it to generate that message, and feed it to the logger. Don’t bloat up and complicate the code base for everyone else just because of your 1-in-1,000 (or more) special case.

Likewise, if you really need to substitute in the output from a system command into your template, use the subprocess-creation features of Java to run the command, collect its output, and feed it into the template. Again, a 1-in-1,000 (or more) special case, no need to clutter up the code base.

In short, it’s potentially dangerous. Make the programmer work a bit in order to do it. Not a whole lot (libraries should and do exist to make it easier), but a bit. Enough to get the programmer hopefully thinking about the consequences of the feature. Don’t just stick it in by default.

But who am I kidding? This is the Java community we are talking about. Too much can never be enough!

Published at 17:27 on 1 September 2023

To give you an idea of the general pathetic hilarity of the situation, I was reviewing some code at work today. It reads in a message from Kafka, obtains a validator object, and calls that object’s isValid() method on the message it receives. That method in return a ValidationResult object, whose valid() method is then called inside an if statement.

This immediately strikes me as odd. When you validate something, it either turns out to be valid or invalid. That’s it. Two options, no more, no less. Yes/no. Black/white. On/off. There is no need to create a new data type to represent a validation result, because a perfectly appropriate data type already exists, built in to Java: the Boolean. Just use that. Far simpler and cleaner.

Maybe the ValidationResult object does something special and has extended features beyond those of a Boolean? Yes, it has a message field! But wait, that field is never accessed. The only thing that is ever done with that object is to call the valid() method, whose purpose is to return the Boolean value that should have been used in the first place.

And what of the validator object? Its class definition is very simple, just one short method that makes some basic checks. If its argument turns out to be invalid, the message part of the result is set to the string “Data is not valid.” No, I am not making this up. Of course the data is not valid, you moron! That is why the valid flag is set to false! This field conveys exactly zero meaningful information.

What other code uses this validating logic? None of it, it turns out! So there was no need for the validator class, either. Could have just added a private isValid() method inside the one (short) source file where this logic is used. Would have been a whole lot clearer, because the person reading the code wouldn’t have had to open another file to determine just what the validation logic is.

So three classes, and three source files, are being used where just one would have sufficed.

Now, this was a particularly egregious example, but this sort of crap-ola happens over and over (and over) again in Java code. Needless complexity everywhere.

Published at 15:15 on 20 August 2023

The above use of StUdLy CaPs courtesy of a community somewhat enamored of them. An overly-cute quirk that for a long time made me shy away from that document preparation system. It’s silly, to be sure, and hardly the No. 1 reason, which is that I had by that time:

• Already had learned troff, which provides the same general functionality,
• Often I did not have access to a laser printer, and troff shares an input language with nroff, which can produce passable output on a simpler and at the time much more common typewriter-like printer. TeX and LaTeX, by contrast, are useless if you don’t have access to a laser printer, phototypesetter, or graphics display.

Time passes. LaTeX grows to be way more popular than poor old troff. Users deveop and share all sorts of macros to do just about anything you want. Those same users have many online forums that can be searched if you get stuck or puzzled. troff gains nothing equivalent. Get frustrated buy troff‘s inability to set text that wraps around illustrations (a standard book publishing technique).

So yeah, it was time. The final push that made spend most of a day reading Knuth’s definitive description of his program was my resumed job search, the desire to make a résumé that breaks a few of the rules (using some of my favourite fonts), and the difficulty of getting good font support for troff. By contrast, there are modern versions of TeX that can read standard font files.

(Yes, I know there are modern versions of troff that can apparently read standard font files. The rub is, they lack certain extensions to standard troff in the version I have been using, extensions that I am making use of, so I would have to rewrite anyhow. Plus. troff still can’t format output by wrapping around illustrations. Why not rewrite in the more powerful and full-featured alternative?)

Anyhow, I’ve now learned enough of LaTeX to make it do what I want to do… for now, which was the goal.