Navigating a sea of repos

From both work and personal projects, I have lots (at the time of writing, 182) repositories cloned on my laptop. Most of these are repositories that I contribute to, though some are other code bases that I’ve cloned to be able to search more effectively with command line tools.

Continue reading

scalafmt in Neovim with scalafmt-native

I previously wrote a post on setting up scalafmt to autoformat Scala code in Neovim. That guide used nailgun to help keep scalafmt fast to run, however my setup was a bit flaky and frequently broke. Fortunately, scalafmt now provides native binaries compiled with GraalVm, which starts up and runs quickly enough that we don’t need to use the more complex nailgun setup. This blog post provides an updated guide on using scalafmt in Neovim with scalafmt-native.

Continue reading

Implicit Classes in Scala

Scala implicit classes allow you to augment the behaviour of existing objects with new functionality. This pattern, sometimes called “Pimp my library”, provides a useful method to implement expressive Scala code.

Continue reading

Global Git Ignores

Git has a great feature that allows you to avoid committing lots of unnecessary files to your repository. By creating a file called .gitignore in your repository, you can declare that files and directories which match certain name patterns should never be added to the repository. This means that, for example, when running Python code from a repostitory, all of the .pyc files generated by Python can be excluded from commits, but will also not be shown when running commands like git status which shows you what local files you have that are uncommitted.

Continue reading

asyncio

At Faculty, we build a lot of our platform’s backend services in Scala. It’s a really nice programming language to work with, and I’ve found the functional programming model and strong typing really effective in writing well tested, robust software, however when it came to writing a lightweight agent for our new jobs feature, we decided that the computational resource demands of running the JVM was too costly.

Continue reading

scalafmt in Neovim

At Faculty, where I work as an engineer developing our data science platform, we write most of our backend services in Scala. To save on time spent discussing code style, we’re trying out autoformatting of our code with scalafmt, with a configuration that reasonably closely reflects our current style.

Continue reading

Data Science APIs: Long Running Tasks

Data science models often require longer running computations for training or predicion, but HTTP requests that take longer than a few seconds to respond are at increased risk of failure. In this post, I’ll show you how to handle longer running tasks in your API with RQ.

Continue reading

Data Science APIs: Building Robust APIs

In the previous post in this series, I covered the basics of building web APIs with Flask. However, without a little care and attention, it’s easy to introduce some unpleasant bugs that could cause your API to fail in unexpected ways or introduce serious security holes. In this blog post I’ll provide a few tips and tricks you can use to guard against these issues.

Continue reading