links

Links to programming related resources I found useful or interesting. A personal "awesome list".

You can find out more about me at: pawroman.dev

All the links are tested after every commit and daily using lychee via GitHub Actions, but if you see anything that's not working, please let me know!

Table of Contents

• Prelude
• Other Lists
• Algorithms
• Architecture and Programming Patterns
• Assorted "How Stuff Works"
• Career Development
• Code Quality and Good Practices
• Command Line Tools
  • GUI tools
• Data
  • Databases
• Design and Web
• Distributed Systems and Cloud Computing
• Docker
• Eye Candy
  • Fonts
  • Themes
• git
  • git help
• Go
• Hardware and CPUs
• Kubernetes
• Linux
• Management / dealing with people
• Networking
• Privacy
• Productivity
• Programming Languages
• Python
  • Python learning
  • Python libraries
  • Python news and community
  • Python tools
• Rust
  • Rust learning
  • Rust libraries
  • Rust news and community
  • Rust posts and talks
  • Rust tools
• Shell
• Testing
  • Static analysis
• Visualization

---

Prelude

Some of my favourite quotes:

"Craftsmanship over Crap"
-- Uncle Bob

"… with proper design, the features come cheaply. This approach is arduous, but continues to succeed."
-- Dennis Ritchie

"Code is a way you treat your coworkers. We interact through the things we make."
-- Michael Feathers

"Theory is when you know everything but nothing works.
Practice is when everything works but no one knows why.
In our lab, theory and practice are combined: nothing works and no one knows why."

---

Other Lists

• Awesome Python
• The Book of Secret Knowledge

Algorithms

Articles:

• Cache replacement policies - Algorithms that tell you how to evict entries from the cache.
• Consistent Hashing
  • The Simple Magic of Consistent Hashing
  • The Ultimate Guide to Consistent Hashing
• Dijkstra's in Disguise
• In Search of an Understandable Consensus Algorithm - (PDF) paper describing the Raft consensus algorithm.
• Introduction to the A* Algorithm - Explained very clearly, with great interactive examples. Also explores BFS and Dijkstra's algorithm.
  • Implementation of A* - A follow-up on the article, with implementation tips.
• Rendezvous hashing
• Reservoir sampling - Sample items from very large or infinite collections.
• The Wavefunction Collapse Algorithm explained very clearly

Books:

• Algorithms by Jeff Erickson
• The Algorithm Design Manual - An excellent resource for all sorts of algorithms, full of references.

Other:

• Big-O Complexity Cheat Sheet
• Raft Consensus Algorithm Visualization - Raft Visualized in a very easy to follow way.
• xoshiro / xoroshiro generators and the PRNG shootout - Small and fast Pseudo-Random Number Generators.

Architecture and Programming Patterns

• Clean Architecture (book)
• Design Patterns - Describes the classic OO patterns.
• Game Programming Patterns - An excellent book on OO patterns found in games. When performance matters.
• System Design Primer - Concepts behind modern(-ish) large-scale systems, with plenty of references.
• The Architecture of Open Source Applications
• The Internals of PostgreSQL - "for database administrators and system developers".
• The Twelve-Factor App - 12 good rules for "service" type software.
• Vertical Slice Architecture - Argues that architecture should be sliced "vertically" (across business domains) rather than "horizontally".
  • PresentationDomainDataLayering - Similar concept from Martin Fowler.
• You are not Google - Words of caution on architecture and scale.

Anti-patterns:

• Database-as-IPC
• Global Variables Are Bad

Visualizing Architecture:

• The C4 model for visualizing software architecture - A promising model for visualizing software.
  • C4-PlantUML - A reasonably decent tool, however the diagrams need to contain few elements to be readable.
  • Visualising software architecture with the C4 model (video) - An introductory talk about this model.

Assorted "How Stuff Works"

• How does UTF-8 turn “😂” into “F09F9882”? - A nice walk-through of UTF-8 internals.

Career Development

• Apprenticeship Patterns - Problem-solution book for career planning, in praise of software craftsmanship. Free to read online.

Code Quality and Good Practices

The Clean trilogy from Robert C. Martin ("Uncle Bob"):

1. Clean Code (book)
2. The Clean Coder (book)
3. Clean Architecture (book)

Programming overall:

• The Future of Programming (video) - A talk from Uncle Bob about the past and the future of programming. Touches on a plethora of subjects, well worth a watch.
• The Pragmatic Programmer (book) - An absolute classic of a book, there's a new 20th Anniversary Edition.
  • Take-home reference - Sums up the book.

Principles:

• AHA Programming - Avoid Hasty Abstractions.
• Don't Repeat Yourself
  • DRY (Don't Repeat Yourself) is a Fallacy - Argues that DRY is mostly about knowledge, not just code.
• Make It Work, Make It Right, Make It Fast
• Principle of least astonishment
• Rule of three - Rule stating that a generalization should only be attempted on third repetition/duplication.
  • The Rule of Three - Explains the rationale and benefits of this principle.

Principle aggregates / manifestos:

• hacker-laws
• Programming Principles
• Software craftsmanship
  • Manifesto for Software Craftsmanship

Code quality measures:

• Cyclomatic complexity

Legacy / Refactoring / Maintenance:

• Avoid rewriting a legacy system from scratch, by strangling it - A very reasonable approach to rolling out new systems, without full rewrites.
• How To Write Unmaintainable Code - Learn what not to do :)
• Keep a Changelog - A set of guidelines for maintaining changelogs (aka. change logs) in projects. This is a very good practice for versioned software.
• Preventing the Collapse of Civilization - A good talk touching on software simplicity and knowledge transfer in software development.
• When costs are nonlinear, keep it small. - Talks about the importance of regular maintenance:

  Do the easy boring job regularly, instead of the hard scary job in a panic.

• Write code that is easy to delete, not easy to extend. - Iterating on code design should be easy.

  Good code isn't about getting it right the first time. Good code is just legacy code that doesn't get in the way.

Other:

• 5 Things I’ve Learned in 20 Years of Programming - Some good advice.
• API Evolution the Right Way
• API Security Checklist
• ARCHITECTURE.md - Advises to document project's architecture in the repo.
• Do Not Log - Argues that in certain cases logging should be avoided over more structured solutions (monitoring, error tracking). Recommends Rust patterns in Python :)
• Empathy is Code Deep - A talk on applying empathy and thinking about others in coding. Code matters and people do too.
• Some Useful Probability Facts for Systems Programming - Probability applies to systems and probabilistic models explain some seemingly weird behaviors rather well.
• Superior Testing: Stop Stopping - Great post on why testing matters (Archived version).
• The AI Hierarchy of Needs - Looking to do AI? Better sort out the basics and foundations first. Contains the "AI pyramid of needs". I would add "Solid engineering" at the very bottom of the pyramid though!

Command Line Tools

• ArchiveBox - Your personal (self-hosted) webpage archive.
• bandwhich - Display current network utilization by process, connection and remote IP/hostname.
• bat - Syntax and git-aware cat clone, with automated paging.
• ctop - top for containers.
• DBCLI - Collection of great CLI database clients, e.g. pgcli, mycli etc.
• direnv - Per dir/project env var management. You can use it to activate Python virtual envs, store development environment credentials, personal access tokens, etc. It "just works" and is fast, compared to some other tools.
• dive - Inspect Docker images, layer by layer. Useful to search for spurious/unneeded files.
• dua - Aka. dua-cli. A (fast!) disk usage analyzer and cleaner. Also has a nice interactive mode.
• eza - Better than ls. Also has a tree-like switch (-T). A maintained fork of exa.
• fd - Better than find.
• gdb-dashboard - Makes GDB so much better.
• git-quick-stats - Get simple statistics for your repo.
• grex - Generate regular expressions matching provided inputs.
• htop - Excellent process viewer.
• HTTPie - cURL for humans. An easy to use CLI HTTP client. Uses Python + requests. Also see xh.
• hyperfine - CLI benchmarking tool.
• iredis - Featureful Redis CLI client with syntax highlight and completions.
• kubectx - Switch between Kubernetes contexts (clusters) and namespaces easily.
• kubetail - Aggregate logs from multiple Kubernetes pods.
• lazydocker - CLI dashboard for Docker and Docker Compose.
• lychee - A fast and featureful link checker.
• Midnight Commander (mc) - The best file manager.
• mosh - Like ssh, but with local echo and roaming. Good for poor connections.
• mtr - Better than ping.
• ncdu - Disk usage analyzer.
• ripgrep - Fast file content searcher.
• rmlint - Free up disk space (e.g. duplicate files).
• sd - A simpler, more intuitive sed replacement.
• speedtest-cli - Speedtest from a command line.
• stern - Tail multiple k8s logs (like kubetail) but more quickly (and optionally in JSON format).
• storm - Manage SSH connections (.ssh/config).
• tealdeer - A very fast tldr client.
• thefuck - Correct the previous command.
• tmux - A terminal multiplexer, more user-friendly than screen.
• tokei - Count the number of source lines, quickly.
• topgrade - Upgrade just about everything using one single command.
• watchexec - Execute a command when a path is modified.
• xh - A HTTPie clone in Rust. Much faster startup time compared to the original.

GUI tools

I'm not a big fan of GUI, but there are a few good ones I use:

• PosteRazor - Print posters (e.g. large images spanning multiple paper pages)
• QDirStat - Analyze disk usage by directory / file type.

Linux Desktop specific:

• Gestures - A simple GUI for libinput-gestures. Easily assign touchpad multi-touch gestures to arbitrary actions. Also has a useful wiki.

Data

Books

• Designing Data-Intensive Applications - Excellent book that acts as a guide to modern data engineering (and distributed systems). Explains the basic concepts, as well as theory and practical concerns. Doesn't get too specific regarding the software vendors. Highly recommended.

Date and time

• Storing UTC is not a silver bullet (archived) - Why "just store all times in UTC" is sometimes not the best approach.
• Your Calendrical Fallacy Is... - A collection of common timekeeping fallacies.

Storage Formats:

• Delta Lake - A promising new "data lake" format, based on parquet. Widely supported across vendors, frameworks and programming languages.

Databases

• Don't Do This (PostgreSQL wiki) - What not to do in Postgres.
• Jepsen Analyses - In-depth analyses of multiple database systems.
• Lessons learned scaling PostgreSQL database to 1.2bn records/month - Very insightful article.
• Nested Set Model - Reasonably efficient solution for storing tree structures in relational databases. Aka. MPTT (also see example implementation in Django: https://github.com/django-mptt/django-mptt)
• PostgreSQL is the worlds' best database - Arguments for PostgreSQL.
  • 10 Things I Hate About PostgreSQL - A rebuttal to the above, listing shortcomings of Postgres.
• Readings in Database Systems (commonly known as the "Red Book") - A short book about databases, with lots of references ("readings").

Design and Web

• 4 Rules for Intuitive UX - Good advice for designing UI, with examples and anti-examples.
• A JavaScript-Free Frontend - You can do without JavaScript (Archived version).
• Brutalist Web Design - A set of very reasonable guidelines for designing websites.
• HTTP headers for the responsible developer - Good practices for HTTP headers.
• Laws of UX - Collection of principles to consider while designing user interfaces.
• My current HTML boilerplate (April 9, 2021) - A useful HTML boilerplate with in-depth explanations.
• Opting your Website out of Google's FLoC Network - How to opt your website out of the newest Google tracking mechanism.

Distributed Systems and Cloud Computing

• Complexity Explained - Complexity science fundamentals, good to keep in mind when dealing with complex systems.
• Nines are not enough: meaningful metrics for clouds - Explains the relationships between SLA, SLI and SLO and how to approach defining them.

Also see the Data section.

Docker

• Broken by default: why you should avoid most Dockerfile examples - Why most Dockerfile examples are broken.
• Intro Guide to Dockerfile Best Practices - Good recommendations for Dockerfiles.

Eye Candy

Fonts

Programming fonts I found the most eye-pleasing and readable.

• Hack - My font of choice. Very easy on the eyes, and very unambiguous.

Themes

• Adapta-gtk-theme - GTK / desktop theme I use. Works great with Cinnamon.

git

• --force considered harmful; understanding git's --force-with-lease - Why git push --force-with-lease is better than git push --force
• delta - A great-looking and featureful CLI git diff viewer.
• Git in a Nutshell - Overview of how git is implemented.
• git-quick-stats - Get simple statistics for your repo.
• Little Things I Like to Do with Git - A few useful tips.

git help

• git-flight-rules - "Lessons learned" guide to git.
• Oh shit, git! - How to revert common git mess-ups.

Go

• The Zen of Go - Go is Go. A few pointers for idiomatic Go code.

Hardware and CPUs

• A fast alternative to the modulo reduction - A clever alternative to modulo for spreading out values across a fixed-sized array, like in a hashmap.
• Amdahl's law - Useful thing to know regarding threading and parallelization.
• Branch prediction - Quick intro to CPU branch prediction.
• Branch Prediction - Fundamentals Every Programmer Need Not Know - Basics and rationale for CPU branch prediction.
• Common Systems Programming Optimizations & Tricks - A few low-level tricks.
• Computer Latency at a Human Scale - Latency with human-relatable scales.
• Floating Point Visually Explained - The best explanation I've ever seen.
• Gallery of Processor Cache Effects - Concise examples showing how cache affects code run time, with good explanations.
• How does your programming language handle “minus zero” (-0.0)? - For numerical programming, it's good to be aware of -0.0 handling.
• How to Quantify Scalability - Some theory, tips and useful models for quantifying systems scalability.
• How x86_64 addresses memory - Memory addressing on this platform is quite complicated.
• Latency Numbers Every Programmer Should Know - Know your latencies.
• Modern Microprocessors - A 90-Minute Guide! - Exceptionally good article on how modern CPUs work. This explains why a lot of low-level optimizations work.
• Performance speed limits - Super-low-level details of how fast can a CPU execute code.
• The Fallacy of Premature Optimization - Avoiding "premature optimization" doesn't mean we shouldn't optimize at all. Thinking about performance early on is important.
• The microarchitecture of Intel, AMD and VIA CPUs. An optimization guide for assembly programmers and compiler makers - (PDF) Very in-depth description of inner workings, ins and outs of various CPUs.
• What Every Programmer Should Know About Memory - (PDF) An in-depth paper on computer memory from (mainly) programming perspective.
• "What The Hardware Does" is not What Your Program Does: Uninitialized Memory - Words of warning not to forget the "abstract machine" that programming languages model.

Latency visualized:

Precision of IEEE 754 Floating Point Values (credit: Wikipedia):

Kubernetes

• An Illustrated Guide To Kubernetes Networking - Slides explaining inner workings of K8S networking.
• CNCF Cloud Native Interactive Landscape - A map grouping all the current and emerging "cloud native" technologies (mostly related to Kubernetes).
  • Cloud Native Trail Map - The migration path to "cloud native".
• k3s - Lightweight Kubernetes distribution.
• Kubernetes Failure Stories - Learn from other people's mistakes.
• The Distributed System ToolKit: Patterns for Composite Containers - Explains Sidecar, Ambassador and Adapter container patterns.
• Two reasons Kubernetes is so complex - The K8S complexity has good reasons behind it.

kubectl quick reference:

• kubectl Cheat Sheet - Handy kubectl cheat sheet
  • Also very useful: kubectl explain -- see: kubectl explain — #HeptioProTip
  • kubectl api-resources -- show all object kinds, their short names and API groups (empty API group means v1).
• API exploration:
  • kubectl explain pod
  • kubectl explain pod.spec.volumes.persistentVolumeClaim -- explain nested specs/objects
• Authorization:
  • kubectl auth can-i -- see: Authorization Overview (k8s docs)
• Generating YAMLs quickly:
  • kubectl create --dry-run -o yaml -- see: kubectl Usage Conventions
• Getting all objects:
  • kubectl get all [-n NAMESPACE] [-l LABEL=VALUE,LABEL2!=VALUE]

Recipes / how-tos:

• Kubernetes Network Policy Recipes - Copy-paste examples of Network Policies with diagrams illustrating their effects.

Linux

• The Linux Kernel documentation - Comprehensive documentation for the Linux kernel.
  • The kernel’s command-line parameters

Management / dealing with people

Books and Resources:

• Turn the Ship Around! - Great book on management from a US Navy captain. Straight to the point, down-to-earth and genuinely interesting (it's a true story and there's a nuclear submarine involved).

  Focuses on leader-leader approach to management and patterns to introduce it in an organization.

• Manager's Playbook - Heuristics for effective management.

Articles / blog posts:

• 22 Principles for Great Product Managers (archived) - Do product management right.
• Commit messages guide - How to write commit messages well.
• Deadlines Are Killing Us, And Almost Everything Else I Know About Leadership - Good article on deadlines, employee motivation, narcissists and management overall.
• Developer to Manager - Collection of interviews with developers who became managers.
• Do the Real Thing - Postulates that in order to achieve success, one should focus on real things, not distractions and fake alternatives.
• Early Evangelists of Agile Engineering Wouldn't Mind Watching It Die - Some well-grounded criticisms of modern "agile".
• Eliminating the Product Owner Role - Argues that the "product owner" role might be spread across a team.
• Habits of High-Functioning Teams - A few principles and rules for high-performing teams.
• Hanlon's razor

  never attribute to malice that which is adequately explained by stupidity.

• How NOT to hire a software engineer - Good advice on what not to focus on during an interview.
• How to be a manager
• How to Deal with Difficult People on Software Projects - We all know "that one guy"...
• How to Deliver Constructive Feedback in Difficult Situations - In praise of Nonviolent Communication (NVC) principles.
• How to Stop Endless Discussions - Glorifies the RFC ("Request For Comments") process.
• Spotify's Failed #SquadGoals - Why the "Spotify agile model" doesn't work, and recommendations for not repeating the same mistakes.

  (...) the Spotify model is revealed as a collection of cross-functional teams with too much autonomy and a poor management structure. Don’t fall for it.

• 'Strong Opinions, Weakly Held' Doesn't Work That Well - Make sure to present your opinions with probability and be willing to accept other opinions; allow the possibility you might be wrong.

  The next time you find yourself making a judgment, don't invoke 'strong opinions, weakly held'. Instead, ask: "how much are you willing to bet on that?" Doing so will jolt people into the types of thinking you want to encourage.

• The Framing of the Developer (archived) - Discusses the "Impact oriented" framing vs "Backlog oriented". Advocates that we should ditch the backlog frame; instead focus on impact and the delivered value.
• The XY problem - When asking people for help, tell them about the problem, before mentioning the attempted solution.
• Top 12 Things That Destroy Developer Productivity
• Unlearning toxic behaviors in a code review culture
• Users hate change - Arguments for not changing your product if it works and giving users a choice.
• What Got You Here Won't Get You There (Book Summary) - Generic tips for leaders.
• When your coworker does great work, tell their manager - There's a lot of work being done that's not being recognized. This is one way to help your coworkers.
• Why does it take so long to build software? - Talks about accidental complexity and rapid growth in volume of software.
• Why software developers (quite honestly) hate Agile - What went wrong with the agile movement and why Software Craftsmanship approach might be better.
• Work is Work - Getting effective in management and getting work done in an organization is hard work. There are some universal principles to follow though.
• Your Company Culture is Who You Hire, Fire, and Promote
  • Part 2: Anatomy of an Asshole
  • Part 3: Breaking Bad: Why Good People Become Evil Bosses

Talks:

• Prioritizing Technical Debt as if Time and Money Matters (video) - Talk about dealing with tech debt in a data-driven way.
• Pushing through Friction (video) - A few tips on pushing through "organizational friction", and getting your ideas across in a large organization.

Timezone tools:

• Every Time Zone - A readable time zone overview. Useful for scheduling meetings.

Estimating time:

• PERT (Program evaluation and review technique)
  • Also see: Three-point estimation

  te = (o + 4m + p) / 6
  sigma_te = (p - o) / 6
  
  Where:
  * o - optimistic estimate (time)
  * m - most likely estimate
  * p - pessimistic estimate
  * te, sigma_te - expected time and its stdev
  

Talking about technical debt with business people:

(From Kent Beck's twitter: https://web.archive.org/web/20191028001516/https://twitter.com/KentBeck/status/1187766114706542593 )

• "Investing in improving structure" over "Paying off technical debt"
• "Resilience & Scaling" over "Technical debt work"
• "Upgrade" over "Refactoring"
• Playing Business cards:
  • Reduce cost (including time cost to market and shipping new features)
  • Protect revenue (including preventing disasters)
  • Increase revenue (selling more, adding new features)

Open Source Programs:

• Open Source Guides For The Enterprise - Resources and guidelines for running an open source program.
• Producing Open Source Software - A book on running open source projects.

Networking

• Endlessh - SSH tarpit server.

• Reserved IP addresses - List of reserved and special purpose addresses (Wikipedia).

  Private IPv4 network ranges:

  10.0.0.0/8     : 10.0.0.0    - 10.255.255.255
  172.16.0.0/12  : 172.16.0.0  - 172.31.255.255
  192.168.0.0/16 : 192.168.0.0 - 192.168.255.255
  

  Other commonly used network ranges:

  Default Docker bridge network
  172.17.0.0/16  : 172.17.0.1   - 172.17.255.255
  

Privacy

Articles

• Nothing to hide argument - A Wikipedia article discussing the "I have nothing to hide" privacy argument.
• Three Reasons Why the "Nothing to Hide" Argument is Flawed

Resources

• Pi-hole - Your custom DNS server, with a lot of features. Can run on a Raspberry Pi.
• privacytools.io - A grab-bag of resources for the privacy-minded.
• WireGuard - The next-generation, fast & easy to use VPN.

Productivity

• Busy Person Patterns - (PDF) How to be less overwhelmed by the lack of time.
• Mental models - A list of mental models to boost your work productivity and efficiency, among other things.
• Mental Models: The Best Way to Make Intelligent Decisions (109 Models Explained)

Programming Languages

Resources for no particular language, useful for comparing them.

• Rosetta Code - Compare language syntax.
• The Computer Language Benchmarks Game - Which are the fastest? Shows why Rust is so great :)

Python

Resources related to the Python programming language.

Note that you should only be using Python 3, as Python 2 will retire (early 2020): Python 2.7 Countdown

Also see: Status of Python branches

Python learning

• Advanced asyncio: Solving Real-world Production Problems - PyCon 2019 (video)
  • Slides
• Fluent Python (book) - Good book if you know Python basics.
• How to Make Mistakes in Python - (PDF). Free book collecting Python antipatterns and common mistakes.
• I Want A New Duck - Advertises typing.Protocol (aka. structural typing) as a way to marry Python's "duck typing" and mypy type checking.
• I'm not feeling the async pressure - Introduction to the concept of backpressure, as well as advice on how to deal with it in Python's asyncio.
• Practical Python (course) - Practical Python course, from one of the legendary Python developers, David Beazley (dabeaz).
• Python 3 Module of the Week - Tutorials for standard library modules.
• Python API Checklist - Quite a few boxes to tick for good Python APIs.
• Python Cheatsheet - A pretty comprehensive cheatsheet.
• PyVideo - A huge collection of Python related videos: conference / meetup talks etc.
• The Hitchhiker's Guide to Python
• Type hinting (and mypy) - PyCon 2019 (video)
  • Blog post (archived)
• Using Asyncio in Python (book) - Worth a read before doing any asyncio.
• wtfpython - A collection of common Python pitfalls and unexpected behaviours.

Python internals and advanced topics:

• CPython internals (video playlist) - Video lectures on Python 2.7 source code. Dated (Python has evolved a bit since 2.7!), but still mostly relevant.
• How Python does Unicode - Learn how str works under the hood and what does len(str) actually return.
• Inside Python dict
• Inside the New GIL - Slides from David Beazley on "new" GIL implementation (introduced in 3.2).
• Modern Dictionaries (video)
• The CPython Bytecode Compiler is Dumb - A post explaining how dumb (simple) it is (non-optimizing).
• What I've Learned About Optimizing Python - Post listing some performance pitfalls related to CPython implementation.

Python libraries

• aio-libs - A collection of asyncio compatible libraries.
• attrs - Alternative to 3.7's dataclasses.
• Beautiful Soup - Pythonic HTML/XML parsing.
• bleach - Sanitize untrusted strings so that they are HTML safe.
• click - Command line argument parser and then some.
• delta-rs (aka. deltalake) - A library for interacting with the Delta Lake format, written in Rust.
• httpx - Requests-inspired HTTP client with both sync and async APIs.
• irc3 - IRC client based on asyncio.
• janus - Queue class that bridges the worlds of threading and asyncio.
• Jinja2 - The only Python template engine worth your time.
• Lark - Fast parsing library. Good alternative to pyparsing.
• lxml - A very fast and featureful (e.g. can use XPath), albeit a bit low-level XML and HTML parsing library.
• NetworkX - Graph ("network") manipulation and analysis library. Comprehensive yet simple and easy to use.
• Numba - JIT compiler for NumPy code. Produces very fast code.
• pathlib - Excellent standard library module for filesystem path manipulation.
• Pendulum - A date/time library with nice API.
• psutil - Swiss army knife for process management and system monitoring.
• pydantic - Pythonic data validation. Supports dataclasses and has a mypy plugin. Perfect for settings/configs and wherever data validation is required.
• pydeps - Visualize Python module dependencies.
• pyjwt - Work with JSON Web Tokens (JWT).
• PyPDF2 - Read and manipulate (e.g. merge) PDF files.
• PySnooper - Poor man's debugger for Python.
• pytest - The only Python test runner worth using these days.
  • Also see my blog post on basic pytest setup
  • pytest-asyncio - Plugin to run asyncio tests easily.
  • pytest-env - Plugin to set environment variables before any code is imported.
• quamash - Use asyncio with Qt event loop.
• requests - The HTTP library for Python.
• tqdm - Nice progress bar library and command line tool.
• WebSockets - asyncio compatible, no-frills WebSocket client/server.
• xdoctest - Better doctest, with pytest integration.
  • Intro video: Developing With Doctests - Using Xdoctest As A Backend
• xmltodict - Extract data from XML just like JSON.
• yarl - URL parsing and manipulation library. Better API (immutable) compared to furl.

Less known, but very useful things in the standard library:

• String constants - Contains ASCII characters in categorized constants.
• types.MappingProxyType - Read-only view into a dict, suitable for constants.

Python news and community

• PyCoder's Weekly newsletter

Python tools

• austin - Profiler / stack frame sampler, with TUI and optional Web UI.
  • Deterministic and Statistical Python Profiling - Introduces the project and describes Python profiling approaches.
• IPython - An enhanced Python shell.
  • ipdb - IPython-enabled debugger.
• line_profiler - Profile Python line-by-line.
• memory_profiler - Profile Python memory usage line-by-line.
• py-spy - A low-overhead sampling Python profiler.
• pyflame - Flame graph profiler based on ptrace (Linux only).
• ruff - An extremely fast Python linter, written in Rust.
• scalene - A high-performance CPU and memory profiler for Python.

Rust

Rust learning

Meta: book of Rust books:

• The Little Book of Rust Books - Aggregates all the Rust related books available freely online.

Quick references / cheatsheets:

• Rust Language Cheat Sheet
• Rust Memory Container Cheat-sheet

Get started with Rust:

• A half-hour to learn Rust - A very quick intro to Rust.
• Clear explanation of Rust's module system - Very good explanation of how modules work.
• easy_rust - Easy Rust. Rust explained using easy English. Also available as a book.
• idiomatic-rust - Aggregates articles, talks and repos which deal with concise, idiomatic Rust.
• Rust by Example - Rust language showcase, demonstrates the features of the language by example.
• Rust Cookbook - How to accomplish common tasks using idiomatic Rust and libraries.
• rust-book-summary - TLDR summary of The Rust Programming Language Book.
• The simplest guide to error handling in Rust - A very concise intro to error handling.
• Tour of Rust - Similar to "Rust by Example", but more approachable and with more narrative.

Intermediate:

• A Quick Look at Trait Objects in Rust - Explains static vs dynamic dispatch (trait objects).
• Accurate mental model for Rust's reference types - Explains what &T and &mut T really are.
• Allocations in Rust - Good, in depth intro to how different memory allocation mechanisms work in Rust.
• Cheap tricks for high-performance Rust - A few easy tricks around compilation parameters to achieve greater performance.
• Context-preserving error handling - How to preserve error context while using thiserror library.
• Creating a Rust function that accepts String or &str - Use Into<String> to accept &str or String without cloning.
• Creating a Rust function that returns a &str or String - Learn to leverage Cow to handle string operations.
• Delete Cargo Integration Tests - A collection of tips towards shorter compile times with tests.
• Error Handling in Rust - Practical guide to handling errors the "raw" Rust way.
• Generic returns in Rust - On handling From/Into traits and how Iterator::collect() works.
• How To Write Fast Rust Code - An insightful story of optimization, including tooling recommendations.
• Join Your Threads - Why ensuring thread join handles are called is important.
• Learn Rust With Entirely Too Many Linked Lists
• Programming Rust - An excellent intermediate-level book on Rust.
• Rust Design Patterns - An interesting collection of Rust patterns, idioms and anti-patterns.
• Rust: A unique perspective - An interesting take on mutability, ownership and borrowing in Rust, using alternative terminology.
• Rust: Structuring and handling errors in 2020 - Describes good error handling practices using anyhow and thiserror.
• rustbattle.net - Rust language quiz.
• Secure Rust Guidelines - Guide for applications with strong security requirements.
• The Edition Guide - Learn about Rust 2018 features.
• The Little Book of Rust Macros - The missing macro tutorial.
• The Rust Performance Book - A collection of Rust performance related tips.
• The Rust Reference - Reference for all language constructs, syntax, types, attributes etc.
• Working with strings in Rust - In-depth article on Rust strings, and their ins and outs, with comparison to C.
• Wrapper Types in Rust: Choosing Your Guarantees - Overview of pointer, reference, cell and synchronous wrapper types.

Advanced:

• Crust of Rust: Subtyping and Variance - (Video) Detailed, practical explanation of type/lifetime covariance, contravariance and invariance. Also covers the basics of PhantomData. All this actually explains a lot of the borrow checker's behavior and errors.
• Crust of Rust: The Drop Check - (Video) A follow up to the above, covering the drop check, which is closely related to PhantomData and type variance.
• min-sized-rust - How to minimize Rust binary size.
• The Rustonomicon
• Writing an OS in Rust (Second Edition)

Evangelism:

• Rewriting the heart of our sync engine - A success story from Dropbox. Very insightful for rewrites overall, not just about Rust.
• Rust Companies - List of companies using Rust in production.
• Rust is for Professionals

  (...) Rust disposes software to a lower defect rate, reduces total development and deployment costs, and is exceptionally satisfying to use.

• Speed of Rust vs C - Rust has some substantial wins over C.
• Sustainability with Rust - Some pros of Rust from the Amazon AWS team.
• Why Discord is switching from Go to Rust - A success story from Discord.
• Why is Rust the Most Loved Programming Language? - Author's personal
• Why Rust - Compares Rust to Go in a few aspects.
• Why Rust for safe systems programming - Reasons for Rust from Microsoft.
  • Follow up to We need a safer systems programming language

Rust libraries

• assert_cmd - Easily test CLI applications.
• camino - UTF-8 Path and PathBuf, for when low-level nitty-gritty details of the OS-specific path encoding don't matter.
• cmd_lib - Makes running shell commands from Rust very succinct.
• humantime - Parse and format human-readable date/times and durations.
• parking_lot - Synchronization primitives faster than standard library.
• PyO3 - Expose Rust libraries to Python, and interact with Python from Rust. Widely used and well-supported.
• reqwest - Async HTTP client library, with a blocking mode. Featureful and well documented.
• rust-phf - Generate hash tables using perfect hash functions at compile time.
• serde - The (de)serialization framework for Rust. Extremely well done.
• static_assertions - Static (compile time) assertions, for API stability and const expressions.
• structopt - Parse CLI args (with env var fallbacks) easily, by defining a struct. Reduces CLI args parsing logic to virually zero. Recently merged into clap.
• wiremock - Excellent HTTP mocking library, e.g. for testing HTTP clients. Async compatible.

Less known, but very useful things in the standard library:

• std::clone::Clone::clone_from - Perform a (possibly optimized) copy-assignment.
• std::mem::discriminant - Compare enum types statically.

Rust news and community

• caniuse.rs - List and search features introduced in each version of Rust (including future ones!).
• Rust Forge - Rust release dates and many useful links related to the project.
• Rust Reddit
• This Week in Rust - Weekly Rust newsletter, archive available online.

Rust posts and talks

• Error Handling Survey - In depth overview and discussion of error handling libraries and approaches. 2019-11-13.
• How I Wrote a Modern C++ Library in Rust
• Is It Time to Rewrite the Operating System in Rust?
• Mocking Time in Async Rust - A neat idea for mocking time in async tests.
• Optimizations That Aren't, Or Are They? - Copy on Write in Rust and C++.
• Read Rust - A large collection of Rust related posts.
• Rust serialization: What's ready for production today? - A quick overview and benchmark of Rust serialization crates. Published 2020-08-28.
• Taking Advantage of Auto-Vectorization in Rust - How to use the type system so the compiler generates SIMD code without using any intrinsics manually.
• The Evolution of a Rust Programmer - Tongue-in-cheek post on different Rust programming styles.
• totally_safe_transmute, line-by-line - A breakdown of a delightful low-level memory access hack, allowing transmute without any unsafe code.

Breaking things:

• Rust in an instant - A cautionary post about monotonic time and linker symbol shadowing (no_mangle).

Rust tools

• cargo-binstall - An effort to bring standardized binary installation to Rust tools. Can save you some compilation time in the CI.
• cargo-bloat - Find out what takes most of the space in your executable.
• cargo-fuzz - Fuzzing Rust code.
• cargo-semver-checks - Check for Rust semver violations.
• std::dbg - Standard library debug macro, to replace ad-hoc println! debugging. Stabilized in 1.32.0.

Shell

• Defensive BASH Programming - Good practices for Bash scripting. (Archived from a defunct blog).
• gitstatus - Faster git status alternative for shells.
• oh-my-zsh - ZSH configuration framework.
• starship - Fast, featureful shell prompt. Compatible with many shells. The new version (0.45.0+) provides a lot more configuration options compared to previous versions.

Testing

• Big List of Naughty Strings - "list of strings which have a high probability of causing issues when used as user-input data"
• Building Good Tests - Walk through good testing practices. Focused on pytest, but also applicable to testing in general.
• Simple Testing Can Prevent Most Critical Failures (paper + video) - An Analysis of Production Failures in Distributed Data-Intensive Systems. Turns out correct error handling and simple testing is enough to prevent most production failures, even in complex distributed systems.
• TDD, Where Did It All Go Wrong (video) - A talk on how to do TDD right.
• Testing Distributed Systems - A curated mega-list of resources related to testing distributed systems.
• Tests that sometimes fail - Good discussion regarding "flaky" tests, enumerating strategies for dealing with them and common causes of flakiness.

Static Analysis

• static-analysis - "A curated list of static analysis tools for all programming languages, config files, build tools, and more."

Visualization

Links related to data visualization.

• Fundamentals of Data Visualization - A book that is meant "as a guide to making visualizations that accurately reflect the data, tell a story, and look professional."
• matplotlib colormaps - Perceptually uniform colormaps (long merged into Matplotlib).
• Stop Using Pie-Charts - Why Pie Charts are unreadable and what do use instead.