
What do you do if you need to run FFmpeg on a fresh install of an operating system; if you spin up a VM, get a new computer, or - as happens a lot more in the world than all of those things combined - you build a docker image? For a lot of OS's, you can get the binaries from your package manager. However sometimes they won't offer them (or, better said, the default repositories won't, as is the case with Amazon Linux) or they might be out of date. What's your next step? I'm sure a lot of folks would say they would build the binaries from source. What does Amazon, one of the biggest companies in the world, suggest you do? Take a look at [this official tutorial](https://aws.amazon.com/blogs/media/processing-user-generated-content-using-aws-lambda-and-ffmpeg/) for AWS Lambda, about a quarter of the way down the page, and you'll see that they direct you to [johnvansickle.com](https://johnvansickle.com/ffmpeg) to get your FFmpeg binary. In fact, if you search something along the lines of "Amazon Linux install FFmpeg", the top few results all direct you to wget or curl from this site.

John hosts static binaries of FFmpeg, built from both the releases (going all the way back to 3.3) *and* the project's master git branch. Maintaining such a robust catalogue of builds is no small thing in itself, but then also serving this to *a lot* of the people who use FFmpeg on AWS is even more of a feat. If you navigate to John's site in your browser, you'll see that he has some donation links. If you're just following that AWS Lambda guide or the top search results I mentioned above, you won't see those links or probably even really consider John's time or effort or contribution to making your life a little easier.

This is probably one of the more minor examples of xkcd 2347, or maybe a slightly altered one where the label at the top is "digital infrastructure by lazy people". If John were to shut his site down who knows how many companies' build and deployment pipelines would just stop. On the other hand, its not terribly difficult to build FFmpeg from source. It would probably be a blip for a lot of engineers, an hour or two of trouble-shooting and maybe a few more to test. There are other things to consider of course - using statically built binaries made by a random guy on the internet isn't a great security practice - but overall its not that big of a deal. The biggest travesty is that whoever writes Amazon's tutorials doesn't care enough to direct the readers to their own [ffmpeg-lambda-layer](https://serverlessrepo.aws.amazon.com/applications/us-east-1/145266761615/ffmpeg-lambda-layer) or to build from source.

Who is John van Sickle anyway?

Who in the world is John Van Sickle?


I've been trying to use Emacs more lately and completely wean myself off VSCode. Projectile is a built-in "project management" (a project in this sense being a codebase) tool which, especially if paired with a language server, enables a lot of really nice functionality like being able to auto-detect your Python venv and navigate the codebase much easier. I had some problems using it for work though. We use a monorepo - which I've come to like after being hesitent about it at first - which has microservices and discreet libraries as subdirectories. Each microservice/library is, for the purposes of Projectile, its own project. The problem was that Projectile was detecting the entire monorepo as a project root, which was less than helpful.

Projectile works by detecting "sentinel files" along the path when you first open a file or directory that is not in an already detected project. Like most Emacs packages, it's written in a literate style that makes it super easy to read through [the source](https://github.com/bbatsov/projectile/blob/master/projectile.el). It looks for the sentinel files following a few different routes, all helpfully explained [here](https://docs.projectile.mx/projectile/projects.html#customizing-project-detection). My problem was that Projectile was skipping the files that *I* wanted it to use to mark the project root and instead hitting the marker at the top of the monorepo. The package provides a few ways to fix this and generally customize how projects are detected: use a `.dir-locals.el` which defines the project root in each directory of the projects that I want to define, put a `.projectile` file at the project roots, or reorder the relative priority of files and how they're used to delineate projects. I opted for the third since it's the easiest by far (since I wanted the changes to apply across lots of different directories). Inserting the snippet below in my init.el fixed all my Projectile problems (I generally work on Python and Elixir stuff, I'll probably need to add a `Cargo.toml` in there for Rust in the future):

```
(custom-set-variables '(projectile-project-root-files-bottom-up '(
    "mix.exs"        ; Elixir mix file
    "pyproject.toml" ; Pyproject file
    ".git"           ; Git VCS root dir
    )))
```

Shoot me an email if you think there's a better way to do this! 

Getting Emac's Projectile to work with monorepos


[xxd](https://linux.die.net/man/1/xxd) is a classic hexdump tool available on most Linux distros that I've encountered. Mostly people use it to inspect hexdumps from binaries, but it has a surprising amount of functionality above and beyond that: it can revert hexdumps to binary, it can *patch* hexdump output back into an existing binary file, and (the subject of this note) it can format its output as a C header file.

`xxd -i ${file}`, where `-i` stands for include, will output the contents of the input file as an array of `unsigned char` followed by the length of the array as an `unsigned int`. I've been using this lately as a quick way to include arbitrary binary data into compiled objects which I then flash onto a microcontroller or another limited system (like with my font for the GameBoy). This makes the binary really easy to address and use in the rest of my code. The downside is that this blows up the header files and can slow down compilation; this hasn't been a problem for me so far since all the projects have been prototypes or PoCs. If I were to need to do this for more mature projects, I would probably use a linker script but the longer I can put off writing linker scripts in a project, the happier I am.

As a side-note: I did a brief search for any kind of history of the xxd utility. It seems to come packaged with Vim, and the creator Tony Nugent is quoted in the [cute quotes part](https://vimdoc.sourceforge.net/htmldoc/quotes.html) of the Vim docs, but I've also seen indications that it existed in a form before it was packaged with Vim. Email me if you have any sources or know more!

Use xxd to create C headers


This site is statically generated with Next.js and hosted on a VPS in a swamp somewhere in New Jersey. It is built out of HTML, CSS, and a handful of home-made React components. The content (posts and notes) are written in markdown with some front matter, which is parsed with using the gray-matter and remark libraries. The code for the site lives in a GitHub repo (which gets deployed to the server through a GitHub action) and the posts and notes are in a separate folder that gets rsync-ed to the server. I have an API route that asks Next.js to regenerate the site, which I call after I have a new post or a certain number of new notes. The site sits behind some Cloudflare shenanigans right now, but maybe that will change (and its probably not necessary).

All Tags

Favorites

Newest

Most Linked