OSS Blog

A 25K commit gift

The other day we celebrated curl reaching 25,000 commits, and just days later I received the following gift in the mail. The text found in that little note is in Swedish and a rough translation to English makes it: Twenty-five thousand thanks for curl. The cake delivery failed so here is a commit mascot and … Continue reading A 25K commit gift →

curl speaks etag

The ETag HTTP response header is an identifier for a specific version of a resource. It lets caches be more efficient and save bandwidth, as a web server does not need to resend a full response if the content has not changed. Additionally, etags help prevent simultaneous updates of a resource from overwriting each other … Continue reading curl speaks etag →

Gwyn Ciesla: A good use of buildah and podman for Fedora packagers

I maintain a lot of packages. Arguably too many. In any case, sometimes I’m working on updating several of them at once. As part of my process, in addition to a local build and usage testing, I run a mock build before committing and submitting to koji. However, you can only run one mock build per release/arch combination per machine at a time. Since I mostly want to build for rawhide, with the hardware I have, I can run 3 builds at a time. 6, if 3 of them are x86_64 and 3 are i686. I can also build on VMs, [..]

Daily web traffic

By late 2019, there’s an estimated amount of ten billion curl installations in the world. Of course this is a rough estimate and depends on how you count etc. There are several billion mobile phones and tablets and a large share of those have multiple installations of curl. Then there all the Windows 10 machines, … Continue reading Daily web traffic →

Abusing PostgreSQL as an SQL beautifier

SQL is query a language that is usually pretty easy to read. However, if people don’t format their queries properly even SQL turn out to be a nightmare. That’s why developers often turn to an SQL beautifier to turn an ugly query into a nicely formatted string. Various tools are available on the web to […] The post Abusing PostgreSQL as an SQL beautifier appeared first on Cybertec.

Debian breaking Unison (again)

Congratulations – Debian/sid now contains a unison binary that is incompatible with Debian/buster, the stable release. That means, everyone who relies on unison for file synchronization across servers (running buster) and development machines (running...

libcurl video tutorial

I’ve watched how my thirteen year old son goes about to acquire information about things online. I am astonished how he time and time again deliberately chooses to get it from a video on YouTube rather than trying to find the best written documentation for whatever he’s looking for. I just have to accept that … Continue reading libcurl video tutorial →

Adventures with Proprietary Software in Disneyland

I'm not a big fan of Disney - I don't like the impact the company has had on copyright or the social messages that they have insinuated over the years. But I have little kids and when I was in the Los Angeles area I knew I had to take them to Disneyland. I have happy childhood memories myself of a visit there. Like my mother did when I was young, I researched everything I could about how to make the trip the most enjoyable for my kids. I planned out a route to get to the rides, what we would likely eat, eve[..]

Richard W.M. Jones: nbdkit new eval plugin and ip filter

nbdkit is our flexible toolkit for building block devices. I just added a couple of new features which will appear in the next stable release, nbdkit 1.18. Previously I’ve talked on this blog and gave a talk at FOSDEM about how you can write block devices in shell script using nbdkit-sh-plugin. But that requires you to use an extra file for the script. What if opening an extra file is too much work? Well now you can specify the script directly on the nbdkit command line using the new eval plugin. You can wr[..]

curl: 25000 commits

This morning I merged pull-request #4651 into the curl repository and it happened to then become the 25000th commit. The first ever public release of curl was uploaded on March 20, 1998. 7924 days ago. 3.15 commits per day on average since inception. These 25000 commits have been authored by 751 different persons. Through the … Continue reading curl: 25000 commits →

Richard W.M. Jones: Short talk about NBD from the KVM Forum 2019

Here’s our short talk about Network Block Device (NBD) given at the KVM Forum last month:

digest 0.6.23: More sha1 refinements

Another new version of digest got onto CRAN earlier today, and was also uploaded to Debian. digest creates hash digests of arbitrary R objects (using the md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, and spookyhash algorithms) permitting easy comparison of R language objects. It is a fairly widely-used package (currently listed at 868k monthly downloads) as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation. This release comes [..]

effective_cache_size: What it means in PostgreSQL

Many PostgreSQL database users might have stumbled over the effective_cache_size parameter in postgresql.conf. But how can it be used to effectively tune the database and how can we speed up PostgreSQL using effective_cache_size? This blog will hopefully answer some of my readers’ questions and reveal the hidden power of this secretive setting. RAM, Linux, PostgreSQL […] The post effective_cache_size: What it means in PostgreSQL appeared first on Cybertec.

ttdo 0.0.4: Extension

A first update release to the still very new (and still very small) ttdo package arrived on CRAN today. Introduced about two months ago in September, the ttdo package extends the most excellent (and very minimal / zero depends) unit testing package tinytest by Mark van der Loo with the very clever and well-done diffobj package by Brodie Gaslam. Just how the package creation was motivated by our needs in teaching STAT 430 at Illinois, so does the extension code in this release which generalized how we extend[..]

Gwyn Ciesla: WT Social

I’m curious if this can actually be a viable replacement for Facebook. If you’re curious, use my invite link! https://wt.social/gi/gwyn-ciesla/friends/8aby

Python 3 deprecation imminent

OSS Journal, November 2026. In less than two month, with the end of the year 2026, Python 3 will be deprecated and will not obtain any further security updates. Despite the announcement of deprecation...

RcppEigen 0.3.3.7.0

A new minor release 0.3.3.7.0 of RcppEigen arrived on CRAN today (and just went to Debian too) bringing support for Eigen 3.3.7 to R. This release comes almost a year after the previous minor release 0.3.3.5.0. Besides the upgrade to the new upstream version, it brings a few accumulated polishes to the some helper and setup functions, and switches to the very nice tinytest package for unit tests; see below for the full list. As before, we carry a few required changes to Eigen in a diff. And as we said befor[..]

Dirk Eddelbuettel: RcppAnnoy 0.0.14

A new minor release of RcppAnnoy is now on CRAN, following the previous 0.0.13 release in September. RcppAnnoy is the Rcpp-based R integration of the nifty Annoy library by Erik Bernhardsson. Annoy is a small and lightweight C++ template header library for very fast approximate nearest neighbours—originally developed to drive the famous Spotify music discovery algorithm. This release once again allows compilation on older compilers. The 0.0.13 release in September brought very efficient 512-bit AVX instruct[..]

curl 7.67.0

There has been 56 days since curl 7.66.0 was released. Here comes 7.67.0! This might not be a release with any significant bells or whistles that will make us recall this date in the future when looking back, but it is still another steady step along the way and thanks to the new things introduced, … Continue reading curl 7.67.0 →

SUSI.AI Smart Speaker release 20191105

After long and painful times due to the switch to buster, we at the FOSSASIA Team finally can release a new stable build of SUSI.AI for our RaspberryPi based smart speaker. SUSI.AI aims at...

Rcpp 1.0.3: More Spit and Polish

The third maintenance release 1.0.3 of Rcpp, following up on the 10th anniversary and the 1.0.0. release both pretty much exactly one year ago, arrived on CRAN yesterday. This deserves a special shoutout to Uwe Ligges who was even more proactive and helpful than usual. Rcpp is a somewhat complex package with many reverse dependencies, and both the initial check tickles one (grandfathered) NOTE, and the reverse dependencies typically invoke a few false positives too. And in both cases did he move the process[..]

Survey: making Getting Things GNOME sustainable as a productivity app for public good

Now that you’ve been introduced to the overall concept of Getting Things Done with the video in my previous blog post, let me show you the secret weapon of chaos warriors who want to follow that methodology with a digital tool they can truly own. Your secret weapon: “Getting Things GNOME” “Getting Things GNOME” is … Continue reading "Survey: making Getting Things GNOME sustainable as a productivity app for public good" The post Survey: making Getting Things GNOME sustainable as a productivity app for public[..]

TeX Live/Debian updates 20191030

Another month, another update of TeX Live in Debian with the usual long list of updated and new packages. The reappearance of turtle graphics via PStricks packages pst-turtle made me laugh. I used turtle...

The goldsmith and the chaos warrior: a typology of workers

As I’ve spent a number of years working for various organizations, big and small, with different types of collaborators and staffers, I’ve devised a simple typology of workers that can help explain the various levels of success, self-organization, productivity and stress of those workers, depending on whether there is a fit between their work type … Continue reading "The goldsmith and the chaos warrior: a typology of workers" The post The goldsmith and the chaos warrior: a typology of workers appeared first[..]

A secret to productivity for busy individuals with chaotic contexts

Over the years, some people have asked me how I manage so many projects—short and long—without forgetting anything, without breaking promises and commitments, all while looking like a zen buddha. A few observers also remarked (often in mockery) that I tend to take a note of everything, that I document an outrageous amount of seemingly … Continue reading "A secret to productivity for busy individuals with chaotic contexts" The post A secret to productivity for busy individuals with chaotic contexts appeared [..]

Calibre 4.2 based on Python3 in Debian/experimental

Following up on the last post on the state of Calibre in Debian, some news for those who want to get rid of Python2 on their computers as soon as possible. With the finally...

Understanding the Rotschild vs GNOME case in 12 minutes

What’s the deal with the Rothschild vs GNOME Shotwell patent litigation case that the GNOME Foundation must defend against, and why does it matter for protecting the Free & Open-Source software community at large? Here’s my personal attempt at explaining the matter with a short video. Please share far and wide. The post Understanding the Rotschild vs GNOME case in 12 minutes appeared first on The Open Sourcerer.

binb 0.0.5: More improvements

The fifth release of the binb package just arrived on CRAN. binb regroups four rather nice themes for writing LaTeX Beamer presentations much more easily in (R)Markdown. As a teaser, a quick demo combining all four themes follows; documentation and examples are in the package. This release contains some nice extensions to the Monash theme by Rob Hyndman](https://robjhyndman.com/). You can see the a longer demo in this pdf and the extended options (i.e. for titlepage) in this pdf. David Selby also correct a [..]

Gwyn Ciesla: podman-compose: Review Request

Want to use docker-compose.yaml files with podman on Fedora? Review podman-compose! I’ll even review one of yours, no extra charge! https://bugzilla.redhat.com/show_bug.cgi?id=1767240

Calling all Tesla owners: let's discuss the source code for the GPLed parts of your car!

A blog post from Software Freedom Conservancy. Blog post by Denver Gingerich. Please email any comments on this entry to . It has been many years since we started working with Tesla to help them resolve their ongoing GPL violations. However, Tesla has still not provided the necessary source code for their cars (a benefit of ownership enshrined in the GPL, which Tesla chooses to use) and the incomplete versions of source they have released are more than 17 months old (at the time of[..]

Pleasures of Tibetan input and typesetting with TeX

Many years ago I decided to learn Tibetan (and the necessary Sanskrit), and enrolled in the university studies of Tibetology in Vienna. Since then I have mostly forgotten Tibetan due to absolute absence of...

Music with the Synthstrom Deluge

I bought a Deluge a while back, and I’ve owned synthesizers and kaossilators and all kinds of other things for years. The Deluge is several things: expensive, awkward to use, but (with practice) it can make some reasonable music. Here … Continue reading →

Dirk Eddelbuettel: littler 0.3.9: More nice new features

The tenth release of littler as a CRAN package is now available, following in the thirteen-ish year history as a package started by Jeff in 2006, and joined by me a few weeks later. littler is the first command-line interface for R as it predates Rscript. It allows for piping as well for shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It also always loaded the methods package which Rscript only started to do more recently. littler lives on Linux and Unix, has[..]

Me, curl and Dagens Nyheter

In the afternoon of October 1st 2019, I had the pleasure of welcoming Linus Larsson and Jonas Lindkvist into my home in Huddinge, south of Stockholm, Sweden. My home is also my office as I work full-time from home. These two fine gentlemen work for Sweden’s largest morning newspaper, Dagens Nyheter, which boasts 850,000 daily … Continue reading Me, curl and Dagens Nyheter →

Prewarming PostgreSQL I/O caches

PostgreSQL uses shared_buffers to cache blocks in memory. The idea is to reduce disk I/O and to speed up the database in the most efficient way possible. During normal operations your database cache will be pretty useful and ensure good response times. However, what happens if your database instance is restarted – for whatever reason? […] The post Prewarming PostgreSQL I/O caches appeared first on Cybertec.

State of Calibre in Debian

To counter some recent FUD spread about Calibre in general and Calibre in Debian in particular, here a concise explanation of the current state. Many might have read my previous post on Calibre as...

dang 0.0.11: Small improvements

A new release of what may be my most minor package, dang, is now on CRAN. The dang package regroups a few functions of mine that had no other home as for example lsos() from a StackOverflow question from 2009 (!!) is one, this overbought/oversold price band plotter from an older blog post is another. More recently added were helpers for data.table to xts conversion and a git repo root finder. Some of these functions (like lsos()) where hanging in my .Rprofile, other just lived in scripts so some years ago I[..]

linl 0.0.4: Now with footer

A new release of our linl package for writing LaTeX letters with (R)markdown just arrived on CRAN. linl makes it easy to write letters in markdown, with some extra bells and whistles thanks to some cleverness chiefly by Aaron. This version now supports a (pdf, png, …) footer along with the already-supported header, thanks to an intiial PR by Michal Bojanowski to which Aaron added nice customization for scale and placement (as supported by LaTeX package wallpaper). I also added support for continued integrat[..]

pkgKitten 0.1.5: Creating R Packages that purr

Another minor release 0.1.5 of pkgKitten just hit on CRAN today, after a break of almost three years. This release provides a few small changes. The default per-package manual page now benefits from a second refinement (building on what was introduced in the 0.1.4 release) in using the Rd macros referring to the DESCRIPTION file rather than duplicating information. Several pull requests fixes sloppy typing in the README.md, NEWS.Rd or manual page—thanks to all contributors for fixing these. Details below. C[..]

digest 0.6.22: More goodies!

A new version of digest arrived at CRAN earlier today, and I just sent an updated package to Debian too. digest creates hash digests of arbitrary R objects (using the md5, sha-1, sha-256, sha-512, crc32, xxhash32, xxhash64, murmur32, and spookyhash algorithms) permitting easy comparison of R language objects. It is a fairly widely-used package (currently listed at 868k monthly downloads) as many tasks may involve caching of objects for which it provides convenient general-purpose hash key generation. This r[..]

R with TensorFlow 2.0 on Debian/sid

I recently posted on getting TensorFlow 2.0 with GPU support running on Debian/sid. At that time I didn’t manage to get the tensorflow package for R running properly. It didn’t need much to get...

Richard W.M. Jones: NBD over AF_VSOCK

How do you talk to a virtual machine from the host? How does the virtual machine talk to the host? In one sense the answer is obvious: virtual machines should be thought of just like regular machines so you use the network. However the connection between host and guest is a bit more special. Suppose you want to pass a host directory up to the guest? You could use NFS, but that’s sucky to set up and you’ll have to fiddle around with firewalls and ports. Suppose you run a guest agent reporting stats back to t[..]

Using American Fuzzy Lop on network clients

Previously I’ve fuzzed hivex and nbdkit using my favourite fuzzing tool, Michał Zalewski’s American Fuzzy Lop (AFL). AFL works by creating test cases which are files on disk, and then feeding those to programs which have been specially compiled so … Continue reading →

Dirk Eddelbuettel: RcppGSL 0.3.7: Fixes and updates

A new release 0.3.7 of RcppGSL is now on CRAN. The RcppGSL package provides an interface from R to the GNU GSL using the Rcpp package. Stephen Wade noticed that we were not actually freeing memory from the GSL vectors and matrices as we set out to do. And he is quite right: a dormant bug, present since the 0.3.0 release, has now been squashed. I had one boolean wrong, and this has now been corrected. I also took the opportunity to switch the vignette to prebuilt mode: Now a pre-made pdf is just included in [..]

What is autovacuum doing to my temporary tables?

Did you know that your temporary tables are not cleaned up by autovacuum? If you did not, consider reading this blog post about PostgreSQL and autovacuum. If you did – well, you can still continue to read this article. Autovacuum cleans tables automatically Since the days of PostgreSQL 8.0, the database has provided this miraculous […] The post What is autovacuum doing to my temporary tables? appeared first on Cybertec.

RIP (for now) Calibre in Debian

The current purge of all Python2 related packages has a direct impact on Calibre. The latest version of Calibre requires Python modules that are not (anymore) available for Python 2, which means that Calibre...

TensorFlow 2.0 with GPU on Debian/sid

Some time ago I have been written about how to get Tensorflow (1.x) running on current Debian/sid back then. It turned out that this isn’t correct anymore and needs an update, so here it...

How PostgreSQL estimates parallel queries

Parallel queries were introduced back in PostgreSQL 9.6, and the feature has been extended ever since. In PostgreSQL 11 and PostgreSQL 12, even more functionality has been added to the database engine. However, there remain some questions related to parallel queries which often pop up during training and which definitely deserve some clarification. Estimating the […] The post How PostgreSQL estimates parallel queries appeared first on Cybertec.

Fixing track_activity_query_size in postgresql.conf

Many of you might have wondered why some system views and monitoring statistics in PostgreSQL can contain incomplete query strings. The answer is that in PostgreSQL, it’s a configuration parameter that determines when a query will be cut off: track_activity_query_size. This blog post explains what this parameter does and how it can be used to […] The post Fixing track_activity_query_size in postgresql.conf appeared first on Cybertec.

10 years in Japan

Exactly 10 years ago, on October 1, 2009, I started my work at the Japan Advanced Institute of Science and Technology (JAIST), arriving the previous day in a place not completely unknown, but with...