Blackcap Blog

Published at 17:52 on 3 December 2017

If you go to the Open Street Map Wiki, you can find pages like this one giving helpful information about how map tiles work, and how tiles map to latitude and longitude.

But you don’t see any mention of a technique I call world pixels, an idea that woke me from a dream last night and which implementing put an end to countless headaches in getting map bounds just right.

Consider the following facts:

• Zoom levels run from 0 through 18.
• Each zoom level, Z, has 2^Z tiles in each direction.
• Each tile has 256 (2⁸) pixels in each direction.
• 18 + 8 = 26.
• 26 ≤ 32.

Therefore, at all zoom levels, it is possible to code the (x, y) coordinates of all pixels in all tiles with two globally unique 32-bit integers. Put the tile pixel dimensions in the 8 least significant bits and the tile numbers in the next 18 most significant bits. Translate your lat/long coordinates to world pixels as early as possible in the process, and all your math then becomes integer math, which runs faster and is free from rounding errors.

Better yet, it makes zooming a breeze. Want to zoom in? Shift left; the most significant bit of the tile pixel number will become the least significant bit of the tile number. Likewise, want to zoom out? Shift right.

Simplicity itself; both easier to understand and it offers faster execution times and it produces more accurate results.

Published at 06:13 on 24 November 2017

Developing a web site to report information in graphical, map form has given me a new appreciation for how much client-side Javascript (a technology I have never much liked to start with) sucks.

In fact, Javascript was never really the first choice for this site. If there was a simpler, purely declarative way to draw a map that used tool tips to display the details of each observation, I’d have used it in a heartbeat. But there doesn’t seem to be, and moreover, the best package I could find for presenting maps used Javascript. So Javascript it was.

The problem is the execution environment for mobile devices. It simply varies too much. For desktop browsers it was fairly easy to come up with a page that worked well with every browser I thought to try. Mobile sites have so far frustrated that desire. What I have works on my iPad, but no friend’s smart phone has made it work likewise. Some don’t let one zoom in at all. Some zoom in but always draw the dots ridiculously small (despite their being specified as 4 pixels in diameter). Some don’t register touches that call the touch callback. And so on.

It shouldn’t have been a surprise, really. It’s not as if most web developers are totally incompetent, yet attempting to use Javascript-heavy web sites on a mobile device is always asking for trouble. Those developers weren’t lazy or incompetent, they were just fighting against a platform that does its best to frustrate portable coding.

The rub is, tool tips are really needed to present as much information as I want to in a single graphic. As such, it’s probably time to realibrate my wants and make Javascript avoidance first and paramount. If presenting that much information requires Javascript, then presenting that much information in one graphic is the wrong goal. Time to come up with a version of the site that is basic, static HTML and point mobile users there.

Published at 10:44 on 21 November 2017

Circa 2011 I dumped Safari for Firefox. Safari had come out with what I term a “turkey upgrade” that made it painfully slow. Plus, Safari’s Javascript engine has always tended to suck. Add to that the recent slowness factor and it was worth putting up with how Firefox’s UI sometimes annoying departs from normal Macintosh standards.

Now Firefox has come out with a turkey upgrade of its own: Firefox Quantum. It’s almost hideously ugly and un-Mac-like. There’s so much clutter up top that the box that you enter the URL to browse to in can almost never display the whole URL at once any more.

Worse, the designers made the atrocious decision to devote the entire top part of the window to the browser tabs; there’s not much place to click on if you just want to move the browser window. You must fight your way to one of the far top corners in order to do that (and “fight” is the correct word, given that it makes it needlessly difficult to move the browser window).

What happens, of course, is that I’m used to Firefox acting how it used to (and, for that matter, how every other Mac program works), where the entire top most part of the window can be used to grab and move it. So I end up grabbing a tab and moving it. Which of course causes the tab to become a window of its own. This is something I almost never want to do. Congratulations, Firefox, you’ve made it easy for me to do something I almost never want and needlessly difficult to do something I often want.

Back to Safari for now.

Published at 22:00 on 28 September 2017

I’ve been using the Eclipse IDE off and on for several years now, mainly because it’s something of a de-facto standard in the Java world, and I want to be something in sync with that world. Well, forget it. I’ve come to the conclusion that the annoyances outweigh the advantages:

1. The editor is nowhere near as powerful as a stand-alone editor. That should not be a surprise; it’s competing for the attention of the Eclipse dev team with so many other priorities. By contrast, the dev teams working on text editor projects are totally focused on making those editors better.
2. The editor is sluggish. It doesn’t respond as promptly to my keystrokes as any dedicated text editor I’ve used. This frequently trips me up.
3. It’s almost as if it has an artificial intelligence engine working away to decide whether or not I’d like auto-completion to be offered for an identifier… and then does precisely the opposite of what I want. The feature pops up unbidden and gets in the way when I’m rapidly typing, yet never appears when I’m paused, trying to remember the precise name of something. Typing Control-Space remedies the latter situation, but still.
4. Also, the auto-complete for things like parenthesis and quotes is thoroughly evil. It’s always introducing syntax errors into my code, because it comes up slightly too late; by the time it’s gratuitously inserting something, I’ve already gotten a keystroke or two in edgewise.
5. The default is to indent with hard tabs, and you have to change many settings to defeat this misfeature.
6. There is no simple, easy way (at least none I’ve found, and I’ve tried) to stop it from making files with lines that have trailing white space.
7. When upgrading to a new version of Eclipse, nothing is done to import settings settings from any previous version you were using.
8. When upgrading to a new version, the new version will create a new directory for its workspaces, instead of seeing if there is any existing such area in a default location used by a previous version. The default workspace path changes from version to version. There is no overriding pattern to the defaults.
9. There’s no way to do something as simple and basic as renaming a project in Eclipse Oxygen. If there is; it’s very well-hidden; the action is not listed in the same menu that creating, deleting, and copying projects are listed.

The straw that broke the camel’s back for me was the mess with the workspace directories. I don’t need three separate, inconsistently-named, locations. Yet that is now what I have. And when I attempted to convert and old workspace into a new one, the conversion failed and left some projects inaccessible. I’m sure there’s a solution that would enable me to fix the problem, but it’s simply not worth continually expending effort at making an overly-complex tool behave itself.

So I’m in the process of reverting to a text editor and using that old reliable standby Ant to do the building of my Java projects. A pity, as some of what Eclipse offers (detection of errors as one types, being able to request auto-completion) really is helpful.

Published at 19:18 on 5 April 2017

I’ve been looking at the Go programming language recently, because (a) it’s started showing up a lot more in job descriptions, and (b) it combines the interesting attributes of both being compiled to machine language and being garbage-collected.

Unfortunately, it also falls victim to what I regard as an unfortunate fad amongst programming language designers in recent years: the decision that full, preemptive threading is something that programs don’t need and that some sort of ersatz (cooperative) concurrency is good enough.

Well, sorry, in my experience it’s just not. Maybe I’m atypically biased, because of my recent experience writing web crawlers, which have to parse pages as part of the job of finding new links to crawl. That’s a computationally complex task that in certain pathological cases can take indefinitely long. The solution is to run the parsing in a thread, and to kill that thread and proclaim the page unusable if it fails to parse in a reasonable amount of time.

This only works with true preemptive threading. The parsing is taking place in some third-party library that’s stuck in a loop someplace. I don’t want to recode the parsing library and clutter it up with dozens of yield statements. I shouldn’t have to recode it. But absent genuine threads, there is no alternative.

Maybe I’m being unfair here; maybe my experience with concurrency is highly unrepresentative. I don’t know, exactly. I can’t imagine I’m the only person who’s tried to make calls into a computationally complex library routine from a multithreaded program. So color me skeptical about the whole “ersatz threads are good enough” mindset.

I’ll note that C# has both full preemptive threading and the ability to be compiled down to machine code (by default, it’s just byte code, but there is what the C# world calls “ahead of time” compilation, which is just what I described). And C# is not a Microsoft-only thing; there’s Mono.

Published at 10:05 on 21 October 2016

ACARS is a digital protocol used by aircraft to transmit messages. It’s been around since the late 1970’s and is decodable using nothing but a sound card and the right software. But, after helping a friend (a technologically-sophisticated one; like me, he has a ham license) who has previously had no luck decoding the messages, it’s clear there’s some tricks involved.

• Don’t use squelch. Squelch will chop off the first tiny fraction of information in a packet, causing decoding errors (typically, messages simply won’t decode). There’s no need to use squelch, anyhow. Squelch exists to prevent humans from being annoyed by listening to the background noise when a frequency is not in use. Computers don’t care about being forced to analyze static, and can easily distinguish between static and an ACARS packet.
• Use a wide bandwidth. A big part of my friend’s problem was that he was using the default AM bandwidth on his communications receiver, which was apparently too narrow. I have myself tried using both the wide and narrow filters on my receiver; only the wide one works. ACARS is apparently a wide-bandwidth mode, and a narrow filter throws away critical information needed to decode a message.
• If using ACARSD, configuration is critical. ACARSD is the most popular freeware package for decoding ACARS. Alas, it’s not exactly user-friendly. To install it you must first configure the installer and manually tell it to create the directories it needs. To configure it you must use a separate program that (re)writes the necessary .INI file. Moreover, that program doesn’t always default to reasonable values as advertised. It claims ACARSD will use the default sound card if none is specified. I found it necessary to explicitly specify the sound card for the default one to be used on my friend’s computer.

Published at 20:25 on 20 September 2016

If you ever install Flash, there it will be. A stupid updater process that pops up and runs whenever you least want such a thing to run, typically when your computer is already busy and slow. And the Adobe Updater is itself a booger-eating fat pig that will pork up resources and make your computer run even slower.

There’s no instructions from Adobe on how to remove it. Of course not; they have delusions of grandeur: they think their software is so important that its updater is more important than you are. In fact, they think it’s about the most important thing there is. Definitely more important than Apple’s own update checks, which manage to run unobtrusively and generally when you’re not using the computer much.

Thankfully, it’s easy enough to remove the darned thing. Just open a shell window and type the following commands:

cd ~/Library/LaunchAgents
launchctl remove `basename com.adobe.ARM.* .plist`
rm com.adobe.ARM.*

Et voilà! The Adobe Updater virus program should no longer run itself automatically.

Not to belabor the obvious, but if you choose to do this, it becomes your responsibility to check for and install updates. And with Flash, that’s important, because Flash security fixes come out all the time.

Thanks to the Life of a Computer Scientist blog for coming up with the solution and posting it.

Published at 23:07 on 30 June 2016

Not really.

I’ve been working on a set of command-line utilities to let me post here without using an interactive browser. Reason is that WordPress is infected with excessive amounts of crap Javascript, to the point that its editor window is nearly useless if one doesn’t have a solid high-speed connection. Which I often don’t while commuting on the ferry.

Anyhow, two idiots have conspired to make my life more difficult than it needs to be. Both have used an object containing actual or implied time zone information to represent an XML-RPC date/time stamp (which doesn’t contain any time zone information).

Idiot No. 1 wrote the WordPress XML-RPC code (or the PHP library that uses same), and Idiot No. 2 wrote the Apache ws-xmlrpc code. Both idiots made feeble and ultimately failing attempts to defeat the lossage their idiocy begat, and I’ve spent most of the evening puzzling out the gyrations necessary to reverse engineer then counteract the lossage caused by both the base design flaw and the ineffectual original countermeasures… on both the client and server ends.

Yes, I’m being uncharitable and abrasive by calling those programmers “idiots”. You would too if your temper had just been worn thin by dealing with bizarre behavior caused by a stupid design decision.

Published at 14:30 on 17 June 2016

This is going to be a very geeky post, but the bug in question just bit me and I am not aware of anyone else having written about it. Worse, the buggy code is actually recommended in the official Python documentation, which claims that the library call:

pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")

Can be replaced with:

pid = subprocess.Popen(["/bin/mycmd", "myarg"]).pid

It can’t. Not unless you want your child processes to mysteriously disappear on you without calls to os.wait() reflecting they’ve completed, that is.

The problem is that the suggested code immediately creates an unreferenced subprocess.Popen object, and this class declares a destructor (i.e. a __del__ method) which automatically reaps exited child processes at GC time. So the code in question creates a race condition as to which code will call os.wait() first: yours, or the destructor.

Arguably, subprocess.Popen should have an option to disable this feature (which is actually the correct behavior if you’re going to hang on to the Popen object and use it to manage the child process). Until such a time the workaround is to do something like:

class PopenNoDel(subprocess.Popen):
  """
  A Popen object that never gratuitously reaps dead children.
  """
  def __del__(self, **kwargs):
    pass

pid = PopenNoDel(["/bin/mycmd", "myarg"]).pid

Thankfully, it didn’t cause me much lost time. I had thought of the recommended code myself, then rejected the idea because of worries about gratuitous process reaping at GC time, and only changed my mind about the idea when the Python manual itself endorsed it. So the cause was fresh in my mind when my child processes started mysteriously vanishing.

Keywords: os.spawn, subprocess.Popen, wait, reap, garbage collector, subprocess, disappear, bug.

Published at 08:21 on 13 May 2016

(more…)