Tuesday, 15 June

15:49

The ARM processor (Thumb-2), part 12: Control transfer [The Old New Thing]

The most basic control transfer is a direct relative branch.

    b       label       ; unconditional branch

The reach of the relative branch is around ±16MB, with a compact 16-bit encoding available for branch targets within 2KB.

The relative branch instruction can be conditionalized on the status flags:

Condition Meaning Evaluation Notes
EQ equal Z = 1  
NE not equal Z = 0  
CS carry set C = 1  
HS high or same unsigned greater than or equal
CC carry clear C = 0  
LO low unsigned less than
MI minus N = 1 signed negative
PL plus N = 0 signed positive or zero
VS overflow set V = 1 signed overflow
VC overflow clear V = 0 no signed overflow
HI high C = 1 and Z = 0 unsigned greater than
LS low or same C = 0 or Z = 1 unsigned less than or equal
GE greater than or equal N = V signed greater than or equal
LT less than N ≠ V signed less than
GT greater than Z = 0 and N = V signed greater than
LE less than or equal Z = 1 or N ≠ V signed less than
AL always always true unconditional

The conditions come in pairs (aside from AL), and toggling the bottom bit negates the condition. For 16-bit conditional branch encoding, this maps to the bottom bit of the first byte of the instruction. For 32-bit conditional branch encoding, you toggle 0x40 in the second byte of the instruction.

The conditions are named after the behavior that is expected if they come directly after a CMP instruction. For example, a BEQ instruction that comes directly after a CMP is a conditional branch that is taken if the comparison was between two equal values.

Four bits of instruction encoding space are lost to encode the condition, so it can reach only 1/16th as far as the unconditional branch: About ±254 bytes for the 16-bit encoding and about ±1MB for the 32-bit encoding.

There are special conditional branch instructions for testing whether a register is zero.

    cbz     Rn, label       ; branch if Rn == 0
    cbnz    Rn, label       ; branch if Rn != 0

These are 16-bit instructions which are available only for low registers, and they are capable only of branching forward by up to 126 bytes.¹

Subroutine calls are performed by branching to the first instruction of the subroutine and putting the return address in the lr register. This should feel familiar, for all of the other non-x86 processors we’ve reviewed perform subroutine linkage the same way.

    ; branch and link, stay in Thumb-2
    bl      label           ; lr = next instruction + 1
                            ; execution resumes at label

    ; branch and link with exchange, switch to classic ARM
    blx     label           ; lr = next instruction + 1
                            ; execution resumes at label

These instructions have a reach of approximately ±16MB.

Windows uses Thumb-2 exclusively, so you won’t see the blx instruction used in this way. The X stands for “exchange”, which means that it swaps between Thumb-2 and classic ARM modes.²

The return address is stored in lr, but with the bottom bit set. There’s a reason for this.

Thumb-2 instructions must be halfword-aligned, and classic ARM instructions must be word-aligned. Therefore, the bottom bit of any code address is known to be zero, so the processor uses it to encode the target instruction set: If the bottom bit is clear, then execution resumes in classic ARM; if the bottom bit is set, then execution resumes in Thumb-2. Switching dynamically between classic ARM and Thumb-2 instruction sets is known as interworking.

Windows uses Thumb-2 exclusively, and the convention is that the bottom bit of function pointers is always set. When you look at function pointers in the debugger, they will always be one larger than the address itself.

    ; branch with exchange
    bx      Rn              ; switch to classic ARM if Rn is even
                            ; execution resumes at Rn & ~1

    ; branch and link with exchange
    blx     Rn              ; lr = next instruction + 1
                            ; switch to classic ARM if Rn is even
                            ; execution resumes at Rn & ~1

Even though the X instructions can switch to classic ARM, that switching feature is never used in Windows. Function pointers always have the bottom bit set, so the destination of the BLX is always Thumb-2.

The last branch instruction is the table-based branch:

    ; table branch byte
    tbb     [Rn, Rm]            ; jump to pc + 2 * (byte at Rn + Rm)

    ; table branch halfword
    tbh     [Rn, Rm, lsl #1]    ; jump to pc + 2 * (halfword at Rn + Rm * 2)

The base register points to the start of a jump table, and the second register is a byte or word index into the table. The value read from the table is then treated as a forward relative branch offset in units of halfwords.

Remember that pc has moved ahead four bytes when the instruction executes, so the forward branch is relative to the next instruction, not to the TBB or TBH instruction.

Since the offsets are stored in an unsigned byte or halfword, the reach of TBB instruction is 514 bytes, and the reach of of the TBH instruction is around 128KB.

One thing you might notice is that, if you assume that the bottom bit of the register is set, these two instructions are equivalent:

    bx      Rn          ; jump to Rn
    mov     pc, Rn      ; jump to Rn

The second version takes advantage of the fact that storing a value into the pc register acts as a control transfer. In practice, you won’t see the MOV version because it takes a 32-bit encoding, whereas BX uses a 16-bit encoding.

Nevertheless, other variations of loading a value into pc are still useful:

    mov     pc, [r0,#4] ; jump to address
    pop     {pc}        ; pop return address and jump there

Popping a value into the instruction pointer is a common pattern. On entry to a function, you push the registers you need to preserve across the call, and on exit you pop them off. The two sets of registers line up, so that everything pops back to the original source register, except that you pop the old lr into pc, so that the pop instruction is a combination “pop registers from the stack” and “return to caller” instruction.

    ; save a bunch of registers, and the return address
    push    {r3-r6,r11,lr}

    ...

    ; restore the registers, except that the return
    ; address goes into pc, thereby jumping there
    pop     {r3-r6,r11,pc}

Next time, we’ll look at conditional execution.

¹ The inability to branch backward with CBNZ explains why the sample atomic sequence we used last time uses a two-instruction sequence of cmp r3, #0 followd by bne: It can’t use cbnz because it wants to branch backward to retry the operation.

² This instruction was clearly named back when there were only two modes. Nowadays, naming the instruction “exchange” would be ambiguous about which of the many modes it is switching to.

The post The ARM processor (Thumb-2), part 12: Control transfer appeared first on The Old New Thing.

15:07

A possible copyright-policy change for glibc [LWN.net]

The GNU C Library developers are asking for comments on a proposal to stop requiring developers to assign their copyrights to the Free Software Foundation. This mirrors the recent change by GCC, except that the community is being consulted first. "The changes to accept patches with or without FSF copyright assignment would be effective on August 2nd, and would apply to all open branches. The glibc stewards, like the GCC SC, continue to affirm the principles of Free Software, and that will never change."

14:21

Aya: writing BPF in Rust [LWN.net]

The first release of the Aya BPF library has been announced; this project allows the writing of BPF programs in the Rust language. "Over the last year I've talked with many folks interested in using eBPF in the Rust community. My goal is to get as many of you involved in the project as possible! Now that the rustc target has been merged, it's time to build a solid foundation so that we can enable developers to write great eBPF enabled apps".

The Big Idea: Christopher Swiedler [Whatever]

Author Christopher Swielder takes a look at what divides not only his characters, but people in our society in his Big Idea for his newest novel, The Orpheus Plot. Read all about how our problems today aren’t so different from a futuristic-space society’s.

CHRISTOPHER SWIEDLER:

It recently occurred to me that I wrote most of The Orpheus Plot between 2016 and 2020. For future generations who might be a little sketchy on early twenty-first century history, this was a) after the invention of the Internet, b) before the COVID-19 pandemic, and c) during the 45th presidency of the United States, when disagreements got so bad that physicists started a petition to replace the term “political polarization” with “political matter/antimatter baryogenesis.”

The Orpheus Plot began with a relatively simple idea: the protagonist, Lucas, is the first kid from the asteroid belt selected to be a cadet in the interplanetary Navy. He’s lived in space his entire life and already knows half of what they’re trying to teach him, but having grown up on a mining ship without a regular school he hasn’t learned half of what the teachers expect him to already know. 

What makes Lucas’s story more complicated is that the relationship between the Navy and the miners of the Belt is already tense and deteriorating rapidly. A big part of the Navy’s job is to enforce customs and mining-rights laws that the Belters are unhappy with. Most of the Navy sees miners as dirty, uneducated, and entirely unsuited for their cadet school. Lucas’s odd position as the only Belter kid on the teaching ship Orpheus makes him a focal point for all of the built-up hostility, and he soon becomes embroiled in a plot to hijack the ship and start a revolution in the Belt.

Developing the motivation for central characters like Lucas is often pretty easy. What’s usually harder is the motivation for the antagonists that oppose them. Characters can (and should!) have flaws and contradictions, but they still need to have a reasonable set of goals and a believable view of the world. As any book on writing will tell you, conflict is the key to storytelling. But for conflict to resonate with the reader, it has to emerge naturally from the characters’ core beliefs. To depict a solar system on the brink of civil war, I needed to develop worldviews for the Navy and the Belters that were both understandable and wholly incompatible. 

Getting back to our present-day mess, one of the most depressing statistics I’ve read recently is that a majority of both political parties now think that the biggest threat to the United States is the people of the other party. If that had been the premise for a sci-fi novel of thirty years ago, people would have called it dystopian if not outright unbelievable. How can the person living on the next street or in the next town be a threat to the survival of your country? Humans are pretty hard-wired to consider otherness a threat, but we’re also social creatures who tend to see everyone around them as part of their identity. When we’re exposed to otherness for long enough our response is to expand our definition of self so that the otherness ceases to exist. Tell a millennial that there was once uproar over the possibility of a Catholic President and they’ll shake their head in disbelief. They understand the difference between Catholicism and Protestantism, but the idea of worrying about it is as silly as caring about whether the President has blond hair.

The problem, unfortunately, is that we’ve stopped being exposed to otherness. We isolate ourselves not geographically but politically, so that the majority of our interactions are with people we already identify with. A century ago, it was virtually impossible for a person to communicate with anyone on the other side of the world. But for the same reasons, it was virtually impossible to not communicate with the ones who lived next door. Technology has made it possible for two people in the same town to develop such different identities that each of them considers the other to be an enemy. 

In a sense, the character of Lucas was a response to this self-sorting and divergence of identity. He is a connecting point between two cultures on the brink of conflict. He believes, like I do, that the two sides of his world see each other as enemies only because they’ve both found ways to segregate themselves. His bravery comes from his insistence that he belongs to both sides and his refusal to accept that there needs to be any kind of division at all.

I’m an optimist about humanity’s future. I believe that people over time find ways to break down barriers them and expand their sense of self. I love science fiction because it lets us imagine all the possible ways our world might evolve, and one of my favorite quotes is a line from Arthur Clarke’s Imperial Earth—an example of both his unfailing optimism and his signature throwaway-quote style—where the U.S. President of the year 2276 bemoans the death of ethnic diversity and how “it will be a pity when we’re all the same shade of off-white.” A pity, yes, but also my hope: that over time we will choose to weave a single social fabric and form an identity that is nothing more, and nothing less, than being human.


The Orpheus Plot: Amazon|Barnes & Noble|Indiebound|Powell’s

Read an excerpt. Visit the author’s site. Follow him on Twitter.

13:28

Communal Computing [Radar]

Home assistants and smart displays are being sold in record numbers, but they are built wrong. They are designed with one person in mind: the owner. These technologies need to fit into the communal spaces where they are placed, like homes and offices. If they don’t fit, they will be unplugged and put away due to lack of trust.

The problems are subtle at first. Your Spotify playlist starts to have recommendations for songs you don’t like. You might see a photo you took on someone else’s digital frame. An Apple TV reminds you of a new episode of a show your partner watches. Guests are asking you to turn on your IoT-enabled lights for them. The wrong person’s name shows up in the Zoom call. Reminders for medication aren’t heard by the person taking the medication. Bank account balances are announced during a gathering of friends.

Would you want your bank account balances announced during a dinner party?

This is the start of a series discussing the design of communal devices–devices designed to work in communal spaces. The series is a call to action for everyone developing communal devices–whether you are creating business cases, designing experiences, or building technology–to take a step back and consider what is really needed.

This first article discusses what communal devices are, and how problems that appear result from our assumptions about how they’re used. Those assumptions were inherited from the world of PCs: the rules that apply to your laptop or your iPad just don’t apply to home assistants and other “smart devices,” from light bulbs to refrigerators.  It isn’t just adding the ability for people to switch accounts. We need a new paradigm for the future of technical infrastructure for our homes and offices. In this series of articles we will tell you how we got here, why it is problematic, and where to go to enable communal computing.

The Wrong Model

Problems with communal devices arise because the industry has focused on a specific model for how these devices are used: a single person buys, sets up, and uses the device. If you bought one of these devices (for example, a smart speaker) recently, how many other people in your household did you involve in setting it up?

Smart screen makers like Amazon and Google continue to make small changes to try to fix the weirdness. They have recently added technology to automatically personalize based on someone’s face or voice. These are temporary fixes that will only be effective until the next special case reveals itself. Until the industry realizes the communal nature of users’ needs they will just be short lived patches. We need to turn the model around to make the devices communal first, rather than communal as an afterthought.

I recently left Facebook Reality Labs, where I was working on the Facebook Portal identity platform, and realized that there was zero discourse about this problem in the wider world of technology. I’ve read through many articles on how to create Alexa skills and attended talks about the use of IoT, and I’ve even made my own voice skills. There was no discussion of the communal impacts of those technologies. If we don’t address the problems this creates, these devices will be relegated to a small number of uses, or unplugged to make room for the next one. The problems were there, just beneath the shiny veneer of new technologies.

Communal began at home

Our home infrastructure was originally communal. Consider a bookcase: someone may have bought it, but anyone in the household could update it with new books or tchotchkes. Guests could walk up to browse the books you had there. It was meant to be shared with the house and those that had access to it.

The old landline in your kitchen is the original communal device.

Same for the old landline that was in the kitchen. When you called, you were calling a household. You didn’t know specifically who would pick up. Anyone who was part of that household could answer. We had protocols for getting the phone from the person who answered the call to the intended recipient. Whoever answered could either yell for someone to pick up the phone elsewhere in the home, or take a message. If the person answering the phone wasn’t a member of the household, it would be odd, and you’d immediately think “wrong number.”

It wasn’t until we had the user model for mainframe time sharing that we started to consider who was using a computer. This evolved into full login systems with passwords, password reset, two factor authentication, biometric authentication, and more. As computers became more common,  what made sense inside of research and academic institutions was repurposed for the office.

In the 1980s and 1990s a lot of homes got their first personal computer. These were shared, communal devices, though more by neglect than by intention. A parent would purchase it and then set it up in the living room so everyone could use it. The account switching model wasn’t added until visual systems like Windows arrived, but account management was poorly designed and rarely used. Everyone just piggybacked on each other’s access. If anyone wanted privacy, they had to lock folders with a password or hide them in an endless hierarchy.

Early Attempts at Communal Computing

Xerox-PARC started to think about what more communal or ubiquitous computing would mean. However, they focused on fast account switching. They were answering the question: how could I get the personal context to this communal device as fast as possible? One project was digitizing the whiteboard, a fundamentally communal device. It was called The Colab and offered a way for anyone to capture content in a meeting room and then walk it around the office to other shared boards.

Not only did the researchers at PARC think about sharing computers for presentations, they also wondered how they could have someone walk up to a computer and have it be configured for them automatically. It was enabled by special cards called “Active Badges,” described in “A New Location Technique for the Active Office.” The paper starts with an important realization:

“…researchers have begun to examine computers that would autonomously change their functionality based on observations of who or what was around them. By determining their context, using input from sensor systems distributed throughout the environment, computing devices could personalize themselves to their current user, adapt their behaviour according to their location, or react to their surroundings.”

Understanding the context around the device is very important in building a system that adapts. At this point, however, researchers were still thinking about a ‘current user’ and their position relative to the system, rather than the many people who could be nearby.

Even Bill Gates had communal technology in his futuristic home back then. He would give every guest a pin to put on their person that would allow them to personalize the lighting, temperature, and music as they went from room to room. Most of these technologies didn’t go anywhere, but they were an attempt at making the infrastructure around us adapt to the people who were in the space.  The term “ubiquitous computing” (also known as “pervasive computing”) was coined to discuss the installation of sensors around a space; the ideas behind ubiquitous computing later led to the Internet of Things (IoT).

Communal Computing Comes Home

When the late 2000s rolled around, we found that everyone wanted their own personal computing device, most likely an iPhone. Shared home PCs started to die. The prevalence of smartphones and personal laptops killed the need for shared home PCs. The drive goal to provide information and communication services conveniently wherever the users happened to be, including if they’re sitting together on their couches.

When the Amazon Echo with Alexa was released, they were sold to individuals with Amazon accounts, but they were clearly communal devices. Anyone could ask their Echo a question, and it would answer. That’s where the problem starts.  Although Echo is a communal device, its user model wasn’t significantly different than the early PCs: one account, one user, shared by everyone in the household. As a result, items being mistakenly ordered by children made Amazon pull back some features that were focused on shopping. Echo’s usage ended up being driven by music and weather.

With the wild success of the Echo and the proliferation of Alexa-enabled devices, there appeared a new device market for home assistants, some just for audio and others with screens. Products from Apple (HomePod with Siri), Google (Home Hub), and Facebook (Portal) followed. This includes less interactive devices like digital picture frames from Nixplay, Skylight, and others.

Ambient Computing

Ambient computing” is a term that has been coined to talk about digital devices blending into the infrastructure of the environment. A recent paper by Map Project Office focused on how “ambient tech brings the outside world into your home in new ways, where information isn’t being channelled solely through your smartphone but rather a series of devices.” We take a step back from screens and wonder how the system itself is the environment.

The concept of ambient computing is related to the focus of marketing organizations on omnichannel experiences. Omnichannel is the fact that people don’t want to start and end experiences on the same device. I might start looking for travel on a smartphone but will not feel comfortable booking a trip until I’m on a laptop. There is different information and experience needed for these devices. When I worked at KAYAK, some people were afraid of buying $1,000 plane tickets on a mobile device, even though they found it there. The small screen made them feel uncomfortable because they didn’t have enough information to make a decision. We found that they wanted to finalize the plans on the desktop.

Ambient computing takes this concept and combines voice-controlled interfaces with sensor interfaces–for example, in devices like automatic shades that close or open based on the temperature. These devices are finding traction, but we can’t forget all of the other communal experiences that already exist in the world:

Device or object Why is this communal?
Home automation and IoT like light bulbs and thermostats  Anyone with home access can use controls on device, home assistants, or personal apps
iRobot’s Roomba People walking by can start or stop a cleaning through the ‘clean’ or ‘home’ buttons
Video displays in office meeting rooms Employees and guests can use the screens for sharing their laptops and video conferencing systems for calling
Digital whiteboards Anyone with access can walk up and start writing
Ticketing machines for public transport All commuters buy and refill stored value cards without logging into an account
Car center screens for entertainment Drivers (owners or borrowers) and passengers can change what they are listening to
Smartphone when two people are watching a video Anyone in arm’s reach can pause playback
Group chat on Slack or Discord People are exchanging information and ideas in a way that is seen by everyone
Even public transportation ticketing machines are communal devices.

All of these have built experience models that need a specific, personal context and rarely consider everyone who could have access to them. To rethink the way that we build these communal devices, it is important that we understand this history and refocus the design on key problems that are not yet solved for communal devices.

Problems with single user devices in the home

After buying a communal device, people notice weirdness or annoyances. They are symptoms of something much larger: core problems and key questions that should have considered the role of communities rather than individuals. Here are some of those questions:

  1. Identity: do we know all of the people who are using the device?
  2. Privacy: are we exposing (or hiding) the right content for all of the people with access?
  3. Security: are we allowing all of the people using the device to do or see what they should and are we protecting the content from people that shouldn’t?
  4. Experience: what is the contextually appropriate display or next action?
  5. Ownership: who owns all of the data and services attached to the device that multiple people are using?

If we don’t address these communal items, users will lose trust in their devices. They will be used for a few key things like checking the weather, but go unused for a majority of the day. They are eventually removed when another, newer device needs the plug. Then the cycle starts again. The problems keep happening and the devices keep getting recycled.

In the following articles we will dive into how these problems manifest themselves across these domains and reframe the system with dos and don’ts for building communal devices.

Raphaël Hertzog: Freexian’s report about Debian Long Term Support, May 2021 [Planet Debian]

A Debian LTS logo

Like each month, have a look at the work funded by Freexian’s Debian LTS offering.

Debian project funding

In May, we again put aside 2100 EUR to fund Debian projects. There was no proposals for new projects received, thus we’re looking forward to receive more projects from various Debian teams! Please do not hesitate to submit a proposal, if there is a project that could benefit from the funding!

We’re looking forward to receive more projects from various Debian teams! Learn more about the rationale behind this initiative in this article.

Debian LTS contributors

In May, 12 contributors have been paid to work on Debian LTS, their reports are available:

  • Abhijith PA did 7.0h (out of 14h assigned and 12h from April), thus carrying over 19h to June.
  • Anton Gladky did 12h (out of 12h assigned).
  • Ben Hutchings did 16h (out of 13.5h assigned plus 4.5h from April), thus is carrying over 2h for June.
  • Chris Lamb did 18h (out of 18h assigned).
  • Holger Levsen‘s work was coordinating/managing the LTS team, he did 5.5h and gave back 6.5h to the pool.
  • Markus Koschany did 15h (out of 29.75h assigned and 15h from April), thus carrying over 29.75h to June.
  • Ola Lundqvist did 12h (out of 12h assigned and 4.5h from April), thus carrying over 4.5h to June.
  • Roberto C. Sánchez did 7.5h (out of 27.5h assigned and 27h from April), and gave back 47h to the pool.
  • Sylvain Beucler did 29.75h (out of 29.75h assigned).
  • Thorsten Alteholz did 29.75h (out of 29.75h assigned).
  • Utkarsh Gupta did 29.75h (out of 29.75h assigned).

Evolution of the situation

In May we released 33 DLAs and mostly skipped our public IRC meeting and the end of the month. In June we’ll have another team meeting using video as lined out on our LTS meeting page.
Also, two months ago we announced that Holger would step back from his coordinator role and today we are announcing that he is back for the time being, until a new coordinator is found.
Finally, we would like to remark once again that we are constantly looking for new contributors. Please contact Holger if you are interested!

The security tracker currently lists 41 packages with a known CVE and the dla-needed.txt file has 21 packages needing an update.

Thanks to our sponsors

Sponsors that joined recently are in bold.

11:42

CodeSOD: A Date With Yourself [The Daily WTF]

Once upon a time, someone wanted to add a banner to a web page. They also wanted the banner to only appear after a certain date. Jack stumbled across their implementation when trying to understand...

10:49

Abstain from abstaining [Seth's Blog]

Even when you’re not completely certain.

Because we can never be certain about the future.

So we show up for the work, do the reading, engage with the problem. The challenge is to find a point of view if we don’t have one yet.

The exception is simple: if, after being well informed, you are willing to accept every outcome, you do us all a favor when you stand down.

Hiding doesn’t help us.

08:14

Urgent: Laws prohibiting algorithmic discrimination [Richard Stallman's Political Notes]

US citizens: call on Zuckerberg to endorse laws prohibiting algorithmic discrimination.

This is a half-measure. Collecting people's personal data endangers them in many ways, and discrimination is only one of them. It also harms society in various ways — for instance, boosting hate groups and conspiracy fantasies; this is not an issue of discrimination by Facebook, even if some of the hate groups might practice or advocate discrimination.

We should not allow anything comparable to Facebook to exist.

Covid-19 in Mexico [Richard Stallman's Political Notes]

In Mexico, 1/4 of the population — 31 million people — have had Covid-19.

I'd expect this to mean around half a million deaths and a larger number with persistent disabilities.

We must protect marine areas [Richard Stallman's Political Notes]

Deep sea explorer Sylvia Earle: we must "protect marine areas and stop eating tuna."

She may be right regarding people in wealthy countries that most seafood we eat is a luxury. But not in poor countries. I've read that around a billion people around the world depend on seafood for their sustenance.

Of course, most of them are not eating expensive tuna. They are eating whatever they can catch near where they live. The people who eat canned herring are not splurging.

I don't want to rush to stop eating tuna in sushi, especially since I only rarely have sushi at all. But I endorse firm measures to end the practice of fishing for tuna, and all forms of overfishing, non sustainable fishing, and ecosystem-damaging fishing.

In some places, "sushi" usually means tuna and salmon only. Other kinds of fish may not even be available. I suggest that people try the other kinds of fish, aside from tuna and salmon, to discover how delicious they are.

Tunisians protesting [Richard Stallman's Political Notes]

Tunisians are protesting after uniformed thugs beat a prisoner to death.

The prisoner was accused of selling illegal drugs. Repression of drug dealers is a bad solution to a gratuitous problem created by prohibition of drugs. The best way to handle drugs that can harm their users is to legalize the drugs and regulate them so that fewer people will use them.

Chilean colonel [Richard Stallman's Political Notes]

Argentina has captured a fugitive Chilean colonel who was one of Chilean dictator Pinochet's murderers.

Map of landmines [Richard Stallman's Political Notes]

*Azerbaijan swaps 15 Armenian PoWs for map of landmines.*

It is very bad to plant antipersonnel landmines.

No appeal on Craig Murray's conviction [Richard Stallman's Political Notes]

The court that convicted Craig Murray for writing about Salmond's trial decided not to let him appeal the peculiar basis given for convicting him.

How can it make sense to let a court decide whether its decision can be appealed? "Is there any chance that you might have misjudged this? No, that's impossible."

Factory conditions spread Covid-19 [Richard Stallman's Political Notes]

Factory working conditions in southeast Asia encourage the spread of Covid-19 among the workers.

Firing school teachers [Richard Stallman's Political Notes]

One candidate for mayor of New York wants to fire most of the public school teachers, leaving only a few "great" teachers, each of whom will "teach" 400 students. >p> This would make sense if students were mass-produced, identical machines.

The most important policy questions [Richard Stallman's Political Notes]

The American people are in strong agreement about the most important policy questions, including preserving democracy. These policies have 2/3 support, and many have 3/4 or 4/5 support. A small minority of extremist Republicans are hijacking the country.

When a protest constitutes a riot [Richard Stallman's Political Notes]

New repressive Republican laws in various states allow thugs to decide whether a protest constitutes a "riot", and to punish anyone nearby when some unidentified person commits a crime.

Heating of summers and winters [Richard Stallman's Political Notes]

Paleontological evidence suggests that global heating may heat both summers and winters the same amount. That would lead to painfully hot summers.

Over 2,000 Palestinians arrested [Richard Stallman's Political Notes]

Israeli thugs have arrested over 2,000 Palestinians, in many cases over obviously bogus charges. Some were protesting; some were at home. Some are journalists, but journalists were arrested despite not being at protests.

Sometimes the thugs attacked protesters with violence and then did not let them get medical treatment.

Sometimes the thugs did not allow volunteer lawyers to meet with the prisoners.

Israel demands block of shipment [Richard Stallman's Political Notes]

Israel demanded that Egypt block the shipment of construction materials needed to rebuild Gaza's destroyed houses, on the grounds that some of it might be used to rebuild HAMAS's military facilities.

Fortunately, Egypt is disregarding that demand. Biden pledged funds to help rebuild Gaza, and must have urged Egypt to allow in the necessary materials to do it.

04:07

Sergio Durigan Junior: I am not on Freenode anymore [Planet Debian]

This is a quick public announcement to say that I am not on the Freenode IRC network anymore. My nickname (sergiodj), which was more than a decade old, has just been deleted (along with every other nickname and channel in that network) from their database today, 2021-06-14.

For your safety, you should assume that everybody you knew at Freenode is not there either, even if you see their nicknames online. Do not trust without verifying. In fact, I would strongly encourage that you do not join Freenode anymore: their new policies are absolutely questionable and their disregard for their users is blatant.

If you would like to chat with me, you can find me at OFTC (preferred) and Libera.

02:42

Ol' Jiggly [QC RSS]

A lot of y'all asked for it, so here!!!!!!! Available for preorder now on my Topatoco store!

01:14

[1149] Kathrin in Trouble pt5 [Twokinds]

Comic for June 14, 2021

In Which a Major Corporation Flirts With My Wife [Whatever]

Here’s how it went down on Twitter today:

Panel One, from me: "I always said the next car I'd get was likely to be electric. Mildly surprised this will be the form it takes. But a) country living means this will actually be practical, b) Krissy was all "hell yes I want that," so." + Picture of the electric Ford f-150. Panel two, from Ford: "We see you, Krissy. Excited for you, @scalzi. Thanks for making a reservation!" + gif of a cute guy in a truck. Panel three, from a reader: "Dude! Ford's totally flirting with your wife!" Panel four, from me: "They're just aware of how good she'll look behind the wheel." + picture of Krissy. Panel five, from Ford: "We just know that good looks run in the Ford family, is all."
John Scalzi

Also, yes, we reserved one of the F-150 Lightnings, i.e., the new electric Ford truck that will come out next year. I wrote a Facebook post explaining why, which I will repost below.

—-

So, in 2015, after I signed that big contract with Tor, one of the things I was going to do was secretly buy Krissy a convertible, as a way of showing my appreciation to her for everything that she had done to help us get to that point — as I’ve frequently said, after all, without Krissy, I absolutely would not have the career that I have had.

I was looking at the Mustangs for this, but when I sneakily brought up convertible Mustangs in conversation to her, she was all, “meh, they’re okay I guess,” and then later just straight up bought a beater convertible from a pal for really cheap just to tool around in for the summer (I mean, really cheap; I have musical instruments that cost more). At that point I admitted to her I had been planning to get her a car but that it hadn’t worked out, so, basically, whenever she decided she wanted a new car, she had a redeemable coupon for one.

In the six years since, she hasn’t really thought to redeem this coupon, until this last week when I was showing her some videos about the upcoming electric Ford F-150, which, aside from having very good range for an electric and a massive closed storage space where the engine would be and huge hauling and towing capacity and more electrical outlets than some apartments (including a 240 V outlet), can also, in the event of a power outage, actually power one’s home for two or three days (with an optional installed power inverter, which of course we would absolutely get). Krissy’s eyes lit up like a house whose power was now being provided by a big-ass truck.

Sooooo now we have a reservation in for a Ford F-150 Lightning, and we are both happy: Krissy because she’s going to get a very cool truck which she will absolutely have a use for out here in the country, and me because I finally get to give her a car (and also because it comes with a bunch of super cool technology stuff which I will totally be a geek for). Expect to see Krissy tooling around in this thing sometime in 2022.

— JS

01:00

A few thoughts on Fuchsia security [OSnews]

I want to say a few words about my current adventure. I joined the Fuchsia project at its inception and worked on the daunting task of building and shipping a brand new open-source operating system.

As my colleague Chris noted, pointing to this comparison of a device running a Linux-based OS vs Fuchsia, making Fuchsia invisible was not an easy feat.

Of course, under the hood, a lot is different. We built a brand new message-passing kernel, new connectivity stacks, component model, file-systems, you name it. And yes, there are a few security things I’m excited about.

Fuchsia is a much bigger deal than most people think. Make no mistake about it – this is the future of all of Google’s end-user facing operating systems, from Chrome OS, Android, all the way down to Wear OS and Google Home devices. The amazing thing is that with the way Fuchsia is built and designed, including its support for Android applications, most users will be none the wiser they’ve jumped from Linux to something new.

PsychDOS: a desktop environment plus extra software for DOS users [OSnews]

The PsychDOS desktop environment is an ANSI-like graphical interface for launching applications and having a few other features. I highly recommend looking at the SCREENSHOTS and DOCS sections, as well as taking a look at the QCKGUIDE.PDF (Page 3.5 Issue #01) file to get a better idea.

I don’t care what anybody thinks – this is an awesome project, and an awesome idea. The readme contains a lot more detailed information about the project.

CuteFish is a new Linux desktop environment [OSnews]

CuteFishOS’s stated goal is to “make a better experience desktop OS”. To do that they’re building a new desktop environment (‘CuteFishDE’) using KDE Frameworks, Qt, and KDE Plasma 5. This desktop will sit at the heart of a new Linux distro called CuteFishOS.

The desktop experience caters to “beginners”, rather than power users. As such, the devs have no (current) plans to add complex, edge-case, or convoluted settings and features. Like Ubuntu, the aim is to provide a basic set of sane defaults that “just work” for most users.

There’s room for a polished, stripped-down Qt alternative to KDE, but I’m not sure if this one is going to be it.

00:14

00:07

[$] quotactl_path() becomes quotactl_fd() [LWN.net]

The quotactl() system call is used to manipulate disk quotas on a filesystem; it can be used to turn quota enforcement on or off, change quotas, retrieve current usage information, and more. The 5.13 merge window brought in a new variant of that system call that was subsequently disabled due to API concerns; its replacement is now taking form.

Monday, 14 June

23:28

Ben Hutchings: Debian LTS work, May 2021 [Planet Debian]

In May I was assigned 13.5 hours of work by Freexian's Debian LTS initiative and carried over 4.5 hours from earlier months. I worked 16 hours and will carry over the remainder.

I finished reviewing the futex code in the PREEMPT_RT patchset for Linux 4.9, and identified several places where it had been mis-merged with the recent futex security fixes. I sent a patch for these upstream, which was accepted and applied in v4.9.268-rt180.

I have continued updating the Linux 4.9 package to later upstream stable versions, and backported some missing security fixes. I have still not made a new upload, but intend to do so this week.

22:56

Link [Scripting News]

Has anyone used &lsqb[double square bracket]] tagging in a blog? If so, I'd love to read some docs about how they did it.

Link [Scripting News]

I'm still watching In Treatment, but season 3 is mostly cringe-worthy. The writing is awful, and the various plots are ridiculous, though Debra Winger is pretty great. It's a very simple concept, they must've gotten bored with it (?) and tried to turn it into some kind of mysterious love affair between patient and doctor and then doctor-as-patient. Also too many patients have miraculous turn-arounds, and it's not clear why. They seem to be charmed by Paul, but we know Paul ain't charming! Oy. I'm slogging through season 3 knowing there's a new season already in progress, with a new shrink played by a familiar actor, and I want to watch it so I don't want to skip anything.

Link [Scripting News]

Poll: Do you prefer white text on a black background (dark mode) or black on white (light mode)?

22:42

Jonathan Dowland: Opinionated IkiWiki v1 [Planet Debian]

It's been more than a year since I wrote about Opinionated IkiWiki, a pre-configured, containerized deployment of Ikiwiki with opinions. My intention was to make something that is easy to get up and running if you are more experienced with containers than IkiWiki.

I haven't yet switched to Opinionated IkiWiki for this site, but that was my goal, and I think it's mature enough now that I can migrate over at some point, so it seems a good time to call it Version 1.0. I have been using it for my own private PIM systems for a while now.

You can pull built images from quay.io, here: https://quay.io/repository/jdowland/opinionated-ikiwiki The source lives here: https://github.com/jmtd/opinionated-ikiwiki A description of some of the changes made to the IkiWiki version lives here: https://github.com/jmtd/ikiwiki/blob/opinionated-doc/README.md

22:35

Origins of COVID-19 [Richard Stallman's Political Notes]

*EU leaders urge unfettered probe into origins of COVID-19.*

I too think that is called for. To achieve it calls for not giving presenting the possibility of a lab leak as wrongdoing by China. It would be a mistake.

Resistant diseases [Richard Stallman's Political Notes]

How should we fund the development of drugs to treat resistant diseases?

This article proposes that governments fund them to do the development, and later are entitled to the drug at a reasonable price. That could be good if all governments will be entitled to the drug at a reasonable price. But I suspect that all but the sponsoring country will have to pay through the nose.

Inviting Big Pharma companies in will surely mean letting them gouge. They will find an excuse, or simply insist.

I think that the drug-manufacturing know-how should be available to any country that either (1) is poor or (2) sponsors drug research appropriately for the size of its economy.

Liberating Covid-19 vaccines [Richard Stallman's Political Notes]

The European Parliament voted to support liberating Covid-19 vaccines. This is non-binding; the parliament has no say over such questions.

Right to state dissent [Richard Stallman's Political Notes]

A UK appeals court endorsed the right to state dissent from established political views about what respect various individuals deserve.

The UK continues to impose censorship in other ways, but this is an important step forward. Unless overturned, it means that people in the UK cannot impose political censorship by law simply by saying, "Your views are harming me!"

Conflicting views [Richard Stallman's Political Notes]

People of Bristol are working on making an accommodation between their conflicting views about Colston, and political values.

I hope that the targets of racism and the targets of economic inequality learn to work together. Together they can win.

Socialist candidate winning election [Richard Stallman's Political Notes]

Pedro Castillo, a socialist candidate, has won the election for president of Peru, subject to reexamination of some of the ballots.

He is not an extremist, and said he intended to continue a market economy, but he wants to raise taxes on extractive foreign companies.

Naturally, the right wing is fabricating charges of fraud, as is its strategy nowadays.

Items polluting the seas [Richard Stallman's Political Notes]

*Single-use bags, plastic bottles, food containers and food wrappers are the four most widespread items polluting the seas.*

Extremist group [Richard Stallman's Political Notes]

Russia labeled the Jehovah's Witnesses as an "extremist group" and has attacked their prayer meetings, sentencing some of them to many years in prison.

This is inexplicable, since they do not advocate violence, or agitate against Putin.

Violating the rule of law [Richard Stallman's Political Notes]

The European Parliament voted to demand sanctions against Hungary and Poland for violating the rule of law.

The wrong those two countries are carrying out consists of infringing the independence of the judiciary.

Unmarked burials of children [Richard Stallman's Political Notes]

The discovery of unmarked burials of indigenous children who died at a Canadian boarding school — which they were forced to attend — has compelled Canada to offer settlement for legal disputes about forcing indigenous children to attend those schools.

The purpose of these schools (which were spread across Canada) was to disconnect indigenous children from their parents, their culture, and their language. That in itself was cruel, but the schools also practiced direct physical forms of cruelty, including giving the students inadequate food.

Canada set up a Truth and Reconciliation Commission to investigate the crimes of those schools, but has not implemented its recommendations.

New gas-powered generator [Richard Stallman's Political Notes]

Australia's planet-roaster government wants to spend 600 million to build a new gas-powered generator that won't be fully utilized, and isn't necessary.

At least, it isn't necessary for providing electricity. I am sure some construction companies and fossil fuel companies desperately need it.

20:42

News Post: A Thousand Scripts [Penny Arcade]

Tycho: Loki is like Wandavision in that it's a kind of puzzle, which is the sort of show I like best. It's also the most optimized use of Disney+ as a platform, because now the hook is set and I sorta have to stick around to find out what this shit is about now. I go hog-ass fucking wild for a Primer, or a Memento. Or even a regular Mento. I should buy some Mentos. I don't surf the Internet correctly. For one thing, I either ronically or i-ronically still use the term surf. I tend to go to individual sites for information; it's not really mediated by feeds or algo shenanigans.…

20:21

Page 39 [Flipside]

Page 39 is done.

19:42

Bringing the power of AI straight to you in our newest bundle! [Humble Bundle Blog]

If the potential and possibility of artificial intelligence has always fascinated you, get ready for the perfect bundle to fill

Continue reading

The post Bringing the power of AI straight to you in our newest bundle! appeared first on Humble Bundle Blog.

18:49

Google's fully homomorphic encryption package [LWN.net]

The Google Developers Blog has this announcement describing the release of a fully homomorphic encryption project under the Apache license. "With FHE, encrypted data can travel across the Internet to a server, where it can be processed without being decrypted. Google’s transpiler will enable developers to write code for any type of basic computation such as simple string processing or math, and run it on encrypted data. The transpiler will transform that code into a version that can run on encrypted data. This then allows developers to create new programming applications that don’t need unencrypted data." See this white paper for more details on how it all works.

18:07

Cicadone [Whatever]

You know what I heard this morning? Nothing! Which is the first for a couple of weeks; the cicadas, the literal background hum of the last fortnight, have mostly gone silent. Because they’re dead, you see. They crawled out of the ground, they mated, they laid eggs, and they died. There are a few stragglers still flying about, but they’re like people at a beach resort as autumn begins; they missed almost all of the fun. I hope they find love anyway.

In any event, even they will be gone in a couple of days, and that will be that until 2038. The nice thing around here, however, is that as the cicadas are going, the fireflies are arriving. They’re much more quiet than the cicadas. Not necessarily prettier — I think the cicadas looked pretty cool, actually — but maybe nicer to gaze at on a summer night. It’s a summer of bugs, it is.

— JS

Upcoming Speaking Engagements [Schneier on Security]

This is a current list of where and when I am scheduled to speak:

The list is maintained on this page.

18:00

Enrico Zini: Pipelining [Planet Debian]

This is part of a series of posts on ideas for an ansible-like provisioning system, implemented in Transilience.

Running actions on a server is nice, but a network round trip for each action is not very efficient. If I need to run a linear sequence of actions, I can stream them all to the server, and then read replies streamed from the server as they get executed.

This technique is called pipelining and one can see it used, for example, in Redis, or Mitogen.

Roles

Ansible has the concept of "Roles" as a series of related tasks: I'll play with that. Here's an example role to install and setup fail2ban:

class Role(role.Role):
    def main(self):
        self.add(builtin.apt(
            name=["fail2ban"],
            state="present",
        ))

        self.add(builtin.copy(
            content=inline("""
                [postfix]
                enabled = true
                [dovecot]
                enabled = true
            """),
            dest="/etc/fail2ban/jail.local",
            owner="root",
            group="root",
            mode=0o644,
        ), name="configure fail2ban")

I prototyped roles as classes, with methods that push actions down the pipeline. If an action fails, all further actions for the same role won't executed, and will be marked as skipped.

Since skipping is applied per-role, it means that I can blissfully stream actions for multiple roles to the server down the same pipe, and errors in one role will stop executing that role and not others. Potentially I can get multiple roles going with a single network round-trip:

#!/usr/bin/python3

import sys
from transilience.system import Mitogen
from transilience.runner import Runner


@Runner.cli
def main():
    system = Mitogen("my server", "ssh", hostname="server.example.org", username="root")

    runner = Runner(system)

    # Send roles to the server
    runner.add_role("general")
    runner.add_role("fail2ban")
    runner.add_role("prosody")

    # Run until all roles are done
    runner.main()

if __name__ == "__main__":
    sys.exit(main())

That looks like a playbook, using Python as glue rather than YAML.

Decision making in roles

Besides filing a series of actions, a role may need to take decisions based on the results of previous actions, or on facts discovered from the server. In that case, we need to wait until the results we need come back from the server, and then decide if we're done or if we want to send more actions down the pipe.

Here's an example role that installs and configures Prosody:

from transilience import actions, role
from transilience.actions import builtin
from .handlers import RestartProsody


class Role(role.Role):
    """
    Set up prosody XMPP server
    """
    def main(self):
        self.add(actions.facts.Platform(), then=self.have_facts)

        self.add(builtin.apt(
            name=["certbot", "python-certbot-apache"],
            state="present",
        ), name="install support packages")

        self.add(builtin.apt(
            name=["prosody", "prosody-modules", "lua-sec", "lua-event", "lua-dbi-sqlite3"],
            state="present",
        ), name="install prosody packages")

    def have_facts(self, facts):
        facts = facts.facts  # Malkovich Malkovich Malkovich!

        domain = facts["domain"]
        ctx = {
            "ansible_domain": domain
        }

        self.add(builtin.command(
            argv=["certbot", "certonly", "-d", f"chat.{domain}", "-n", "--apache"],
            creates=f"/etc/letsencrypt/live/chat.{domain}/fullchain.pem"
        ), name="obtain chat certificate")

        with self.notify(RestartProsody):
            self.add(builtin.copy(
                content=self.template_engine.render_file("roles/prosody/templates/prosody.cfg.lua", ctx),
                dest="/etc/prosody/prosody.cfg.lua",
            ), name="write prosody configuration")

            self.add(builtin.copy(
                src="roles/prosody/templates/firewall-ruleset.pfw",
                dest="/etc/prosody/firewall-ruleset.pfw",
            ), name="write prosody firewall")

    # ...

This files some general actions down the pipe, with a hook that says: when the results of this action come back, run self.have_facts().

At that point, the role can use the results to build certbot command lines, render prosody's configuration from Jinja2 templates, and use the results to file further action down the pipe.

Note that this way, while the server is potentially still busy installing prosody, we're already streaming prosody's configuration to it.

If anything goes wrong with the installation of prosody's package, the role will be marked as failed and all further actions of the same role, even those filed by have_facts() will be skipped.

Notify and handlers

In the previous example self.notify() also appears: that's my attempt to model the equivalent of Ansible's handlers. If any of the actions inside the with produce changes, then the RestartProsody role will be executed, potentially filing more actions ad the end of the playbook.

The runner will take care of collecting all the triggered role classes in a set, which discards duplicates, and then running the main() method of all resulting roles, which will cause more actions to be filed down the pipe.

Action conditions

Sometimes some actions are only meaningful as consequences of other actions. Let's take, for example, enabling buster-backports as an extra apt source:

        a = self.add(builtin.copy(
            owner="root",
            group="root",
            mode=0o644,
            dest="/etc/apt/sources.list.d/debian-buster-backports.list",
            content="deb [arch=amd64] https://mirrors.gandi.net/debian/ buster-backports main contrib",
        ), name="enable backports")

        self.add(builtin.apt(
            update_cache=True
        ), name="update after enabling backports",
           # Run only if the previous copy changed anything
           when={a: ResultState.CHANGED},
        )

Here we want to update Apt's cache, which is a slow operation, only after we actually write /etc/apt/sources.list.d/debian-buster-backports.list. If the file was already there from a previous run, we can skip downloading the new package lists.

The when= attributes adds an annotation to the action that is sent town the pipeline, that says that it should only be run if the state of a previous action matches the given one.

In this case, when on the remote it's the turn of "update after enabling backports", it gets skipped unless the state of the previous "enable backports" action is CHANGED.

Effects of pipelining

I ported enough of Ansible's modules to be able to run the provisioning scripts of my VPS entirely via ansible.

This is the playbook run as plain Ansible:

$ time ansible-playbook vps.yaml
[...]
servername       : ok=55   changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

real    2m10.072s
user    0m33.149s
sys 0m10.379s

This is the same playbook run with Ansible speeded up via the Mitogen backend, which makes Ansible more bearable:

$ export ANSIBLE_STRATEGY=mitogen_linear
$ time ansible-playbook vps.yaml
[...]
servername       : ok=55   changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

real    0m24.428s
user    0m8.479s
sys 0m1.894s

This is the same playbook ported to Transilience:

$ time ./provision
[...]
real    0m2.585s
user    0m0.659s
sys 0m0.034s

Doing nothing went from 2 minutes down to 3 seconds!

That's the kind of running time that finally makes me comfortable with maintaining my VPS by editing the playbook only, and never logging in to mess with the system configuration by hand!

Next steps

I'm quite happy with what I have: I can now maintain my VPS with a simple script with quick iterative cycles.

I might use it to develop new playbooks, and port them to ansible only when they're tested and need to be shared with infrastructure that needs to rely on something more solid and battle tested than a prototype provisioning system.

I might also keep working on it as I have more interesting ideas that I'd like to try. I feel like Ansible reached some architectural limits that are hard to overcome without a major redesign, and are in many way hardcoded in its playbook configuration. It's nice to be able to try out new designs without that baggage.

I'd love it if even just the library of Transilience actions could grow, and gain widespread use. Ansible modules standardized a set of management operations, that I think became the way people think about system management, and should really be broadly available outside of Ansible.

If you are interesting in playing with Transilience, such as:

  • polishing the packaging, adding a setup.py, publishing to PIP, packaging in Debian
  • adding example playbooks
  • porting more Ansible modules to Transilience actions
  • improving the command line interface
  • test other ways to feed actions to pipelines
  • test other pipeline primitives
  • add backends besides Local and Mitogen
  • prototype a parser to turn a subsets of YAML playbook syntax into transilience actions
  • adopt it into your multinational organization infrastructure to speed up provisioning times by orders of magnitude at the cost of the development time that it takes to turn this prototype into something solid and road tested
  • create a startup and get millions in venture capital to disrupt the provisioning ecosystem

do get in touch or send a pull request! :)

17:21

Security updates for Monday [LWN.net]

Security updates have been issued by Arch Linux (apache, gitlab, inetutils, isync, kube-apiserver, nettle, polkit, python-urllib3, python-websockets, thunderbird, and wireshark-cli), Debian (squid3), Fedora (glibc, libxml2, mingw-openjpeg2, and openjpeg2), Mageia (djvulibre, docker-containerd, exif, gnuchess, irssi, jasper, kernel, kernel-linus, microcode, python-lxml, python-pygments, rust, slurm, and wpa_supplicant, hostapd), openSUSE (389-ds and pam_radius), Oracle (.NET Core 3.1, container-tools:3.0, container-tools:ol8, krb5, microcode_ctl, postgresql:12, postgresql:13, and runc), Red Hat (dhcp, postgresql, postgresql:10, postgresql:12, postgresql:9.6, rh-postgresql10-postgresql, rh-postgresql12-postgresql, and rh-postgresql13-postgresql), Scientific Linux (dhcp and microcode_ctl), SUSE (ardana-neutron, ardana-swift, cassandra, crowbar-openstack, grafana, kibana, openstack-dashboard, openstack-ironic, openstack-neutron, openstack-neutron-gbp, openstack-nova, python-Django1, python-py, python-pysaml2, python-xmlschema, rubygem-activerecord-session_store, venv-openstack-keystone, crowbar-openstack, grafana, kibana, monasca-installer, python-Django, python-py, rubygem-activerecord-session_store, freeradius-server, libjpeg-turbo, spice, and squid), and Ubuntu (rpcbind).

16:35

Pluralistic: 14 Jun 2021 [Pluralistic: Daily links from Cory Doctorow]


Today's links



American slums of yesteryear.

The Rent's Too Damned High (permalink)

This week on my podcast, I read my Medium column "The Rent's Too Damned High," an essay about the economic incoherence of making home-ownership the path to economic security and a middle-class life.

https://medium.com/p/520f958d5ec5/

Obviously, home-ownership does allow for intergenerational wealth accumulation (simply compare the wealth accumulation gap between Black families who were excluded from home subsidies like the GI Bill and white families who weren't), but that's not the whole story.

Owning a home is a good way to insulate yourself from a landlord's whims and a good place to stash gains from the labor market, but when a nation turns to asset appreciation – not labor – to increase prosperity, almost everyone loses within a generation or two.

Without strong labor rights, guaranteed pensions are replaced with a mandate to pump your savings into the stock market casino, where you will always be the sucker at the table. The insufficiency of America's 401k-based retirement savings is grimly comic.

When workers surrendered employment-based security for asset-based security, we lost the bloc that fought for adequate Social Security and affordable tuition. That means that your parents' house needs to fund their retirement and your college tuition.

It's likely not adequate for both, but even it is, it's not going to be enough to pay for your own downpayment, which you'll need if your parents are going to liquidate their sole major asset to keep themselves from burdening you while you enter the labor market.

That means homeowners' kids are overwhelmingly likely to end up tenants, and that's the second whammy. You see, the most reliable way to increase the value of owning a house is to make tenancy worse.

The fact that the home your parents bought for $30k is worth $1.5m today can't be explained by the new roof, the finished basement, the cool neighborhood boutiques or the people who want to move to your hometown for work.

The main driver of real-estate appreciation is that being a homeowner is better than being a tenant – so everything that makes tenancy worse makes homeowning better, which makes houses worth more.

It's not just tax-advantaged shelter (homeowners can write off half their monthly shelter bills, tenants can't). It's also the fact that every time a house sells, the market rate is set by bidders who are buying the house as "income property" – that is, to rent it out.

Just as in labor markets, shelter markets are zero sum. When you have the income security that comes with the right to a regular shift, your boss has the cost-insecurity of having to pay you when business is slow.

Likewise, if you have the right to force your landlord to fix the roof, the rental is worth less. If your landlord can quickly evict you when a better tenant comes along, the rental is worth more. If rent increases are set by law, the rental is worth less.

If your landlord can charge usury interest, or force you to use an ISP that pays a kickback, the rental is worth more.

Everything that's better for tenants is worse for landlords – and the worse things are for landlords, the less they're willing to spend on income property.

The more a landlord is willing to spend to buy a house, the more everyone else has to spend to outbid them. Everything that makes life worse for tenants makes homes more valuable.

Convincing America that home ownership would increase the middle-class also created a large constituency who'd simp for landlords in order to increase the value of their single major asset.

The joke was on us. Now that Wall Street is using the trillions the Trump stimulus pumped into the financial markets to pay 15% above asking, in cash, for every single-family dwelling that hits the market, everyone who doesn't have a home won't ever get one.

All those home owners' kids will be tenants, and will be subject to every depredation that home owners backed in order to protect their kids. Karma's a nasty thing.

America once had strong labor markets, whose dividends were turned into sturdy homes. When we threw in our lot with our bosses on labor rights, we walked into a trap that would cost us our homes, too.

Here's the podcast episode:

https://craphound.com/news/2021/06/13/the-rents-too-damned-high/

and here's the MP3 (hosting courtesy of the Internet Archive, who will host all your stuff for free, forever):

https://archive.org/download/Cory_Doctorow_Podcast_392/Cory_Doctorow_Podcast_392_-_The_Rents_Too_Damned_High.mp3

Here's the RSS for my podcast:

http://feeds.feedburner.com/doctorow_podcast

and here's that original article:

https://gen.medium.com/the-rents-too-damned-high-520f958d5ec5



Collapsed bridge; Harris & Ewing, photographer; between 1921 and 1923; Library of Congress.

Highway to Hell (permalink)

The infrastructure bills are working their way through Congress, and Republicans are indiscriminately blocking them. Take the surface transportation bills: $547b over five years, that passed with only one GOP vote in favor.

That might seem like a lot, but it's just a re-authorization of existing spending. It doesn't authorize a single cent of new maintenance and upkeep – it just continues the existing level of spending. Without it, America would stop maintaining its infrastructure altogether.

In other words, the entire GOP caucus (except Brian Fitzpatrick, R-PA) voted to zero out America's infrastructure maintenance programs for the next five years.

https://prospect.org/politics/everything-you-need-to-know-about-the-infrastructure-bills/

But don't get too smug. Congressional and Senate Dems are also playing games with infrastructure, voting to let America's rail, highways and bridges continue to crumble.

Take the bipartisan "Problem Solvers Caucus," composed of Republicans who think they're still in power and Democrats who wish Republicans were still in power.

They've announced a "compromise" infrastructure bill worth $1.125T – but it's a scam.

That sum includes the existing budget for infrastructure (the inadequate sum that the GOP just voted against), bringing the true total down to $761.8b over five years.

This pattern – inadequate sums that are even lower than they seem at first, thanks to deceptive accounting – is repeated in every "bipartisan" effort to create an infrastructure budget.

The Senate "gang" of 5 Dems and 5 Repubs claim they've agreed to $1.2T in spending, but only $579B of that is new spending.

https://www.wsj.com/articles/bipartisan-group-of-senators-reaches-agreement-on-infrastructure-proposal-11623360255

That Senate deal is such dogshit that it would be laughable, except that it's being treated as serious. For one thing, it claws back covid money that's been allocated to the states…for infrastructure spending, and it imposes a tax on electric vehicle drivers.

To top it all off, McConnell has said there's "no way" the GOP will support it. To sum up: it's a wholly inadequate sum that confiscates state infrastructure budgets and depends on taxing people for switching to electric cars, and it's got no chance of passing.

And they call us unrealistic!

There is no chance of getting the spending America needs through Congress with "regular order." If the agenda Americans voted for is to be realized, Dems will have to abandon bipartisanship.

Unfortunately, the Dem establishment are basically Republicans. As AOC told CNN, the betrayal is coming from inside the House (or rather, the Senate).

https://www.commondreams.org/news/2021/06/13/ocasio-cortez-says-elephant-room-senate-democrats-blocking-their-own-partys-agenda

"I do think that we need to talk about the elephant in the room, which is Senate Democrats blocking crucial items in a Democratic agenda for reasons that I don't think hold a lot of water."

"Do we settle for an infrastructure package that has been largely designed by Republicans in order to get 60 votes, or can we really transform this country, create millions of union jobs, revamp our power grid, get people's bridges fixed and schools rebuilt?"

AOC singled out Joe Manchin for taking positions that align with the Koch network and dark-money priorities: "There's a reason the Koch brothers are really doing victory laps about Joe Manchin's opposition to the filibuster."



This day in history (permalink)

#20yrsago ORBS spam blocklist goes dark https://web.archive.org/web/20010618061402/https://www.salon.com/tech/feature/2001/06/08/orbs/index.html

#10yrsago WIPO boss: fair use is a "negative agenda," rights for blind people are a distraction, let's make more copyright! https://memex.craphound.com/2011/06/14/head-of-un-copyright-agency-says-fair-use-is-a-negative-agenda-wants-to-get-rid-of-discussions-on-rights-for-blind-people-and-go-back-to-giving-privileges-to-giant-companies/

#5yrsago Steeplejack: diverse YA fantasy driven by expert plotting https://memex.craphound.com/2016/06/14/steeplejack-diverse-ya-fantasy-driven-by-expert-plotting/

#5yrsago Appeals court: FCC has jurisdiction to impose net neutrality on ISPs https://www.vice.com/en/article/8q85n4/net-neutrality-appeals-court

#5yrsago Cable industry wants you to know that competition is bad for its customers https://www.techdirt.com/articles/20160610/08243434680/cable-industry-proclaims-more-competition-hurts-consumers-damages-economic-efficiency.shtml

#5yrsago Goldman Sachs bribed Libyan officials with sex workers, private jet rides, then lost all their money https://www.theguardian.com/business/2016/jun/13/goldman-sachs-hired-prostitutes-to-win-libyan-business-court-told

#5yrsago Peter Thiel’s lawyer threatens Gawker for talking about Donald Trump’s “hair” https://gawker.com/now-peter-thiels-lawyer-wants-to-silence-reporting-on-t-1781918385

#5yrsago Air Force loses access to database tracking fraud investigations to 2004 https://arstechnica.com/information-technology/2016/06/database-corruption-erases-100000-air-force-investigation-records/

#1yrago LA schools returned grenade launchers, kept assault rifles https://pluralistic.net/2020/06/14/with-nnedi-today-4pm/#leso



Colophon (permalink)

Today's top sources: Naked Capitalism (https://www.nakedcapitalism.com/).

Currently writing:

  • Spill, a Little Brother short story about pipeline protests. Friday's progress: 264 words (5483 words total).
  • A Little Brother short story about remote invigilation. PLANNING

  • A nonfiction book about excessive buyer-power in the arts, co-written with Rebecca Giblin, "The Shakedown." FINAL EDITS

  • A post-GND utopian novel, "The Lost Cause." FINISHED

  • A cyberpunk noir thriller novel, "Red Team Blues." FINISHED

Currently reading: Analogia by George Dyson.

Latest podcast: How To Destroy Surveillance Capitalism (Part 06) https://craphound.com/nonficbooks/destroy/2021/05/10/how-to-destroy-surveillance-capitalism-part-06/
Upcoming appearances:

Recent appearances:

Latest book:

Upcoming books:

  • The Shakedown, with Rebecca Giblin, nonfiction/business/politics, Beacon Press 2022

This work licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.


How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/web/accounts/303320

Medium (no ads, paywalled):

https://doctorow.medium.com/

(Latest Medium column: "Illegitimate Greatness," on what we can learn from Ida M Tarbell's century-old critique of John D Rockefeller https://doctorow.medium.com/illegitimate-greatness-674353e7cdf9)

Twitter (mass-scale, unrestricted, third-party surveillance and advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla

TikTok Can Now Collect Biometric Data [Schneier on Security]

This is probably worth paying attention to:

A change to TikTok’s U.S. privacy policy on Wednesday introduced a new section that says the social video app “may collect biometric identifiers and biometric information” from its users’ content. This includes things like “faceprints and voiceprints,” the policy explained. Reached for comment, TikTok could not confirm what product developments necessitated the addition of biometric data to its list of disclosures about the information it automatically collects from users, but said it would ask for consent in the case such data collection practices began.

16:28

Enrico Zini: Use ansible actions in a script [Planet Debian]

This is part of a series of posts on ideas for an ansible-like provisioning system, implemented in Transilience.

I like many of the modules provided with Ansible: they are convenient, platform-independent implementations of common provisioning steps. They'd be fantastic to have in a library that I could use in normal programs.

This doesn't look easy to do with Ansible code as it is. Also, the code quality of various Ansible modules doesn't fit something I'd want in a standard library of cross-platform provisioning functions.

Modeling Actions

I want to keep the declarative, idempotent aspect of describing actions on a system. A good place to start could be a hierarchy of dataclasses that hold the same parameters as ansible modules, plus a run() method that performs the action:

@dataclass
class Action:
    """
    Base class for all action implementations.

    An Action is the equivalent of an ansible module: a declarative
    representation of an idempotent operation on a system.

    An Action can be run immediately, or serialized, sent to a remote system,
    run, and sent back with its results.
    """
    uuid: str = field(default_factory=lambda: str(uuid.uuid4()))
    result: Result = field(default_factory=Result)

    def summary(self):
        """
        Return a short text description of this action
        """
        return self.__class__.__name__

    def run(self, system: transilience.system.System):
        """
        Perform the action
        """
        self.result.state = ResultState.NOOP

I like that Ansible tasks have names, and I hate having to give names to trivial tasks like "Create directory /foo/bar", so I added a summary() method so that trivial tasks like that can take care of naming themselves.

Dataclasses allow to introspect fields and annotate them with extra metadata, and together with docstrings, I can make actions reasonably self-documeting.

I ported some of Ansible's modules over: see complete list in the git repository.

Running Actions in a script

With a bit of glue code I can now run Ansible-style functions from a plain Python script:

#!/usr/bin/python3

from transilience.runner import Script

script = Script()

for i in range(10):
    script.builtin.file(state="touch", path=f"/tmp/test{i}")

Running Actions remotely

Dataclasses have an asdict function that makes them trivially serializable. If their members stick to data types that can be serialized with Mitogen and the run implementation doesn't use non-pure, non-stdlib Python modules, then I can trivially run actions on all sorts of remote systems using Mitogen:

#!/usr/bin/python3

from transilience.runner import Script
from transilience.system import Mitogen

script = Script(system=Mitogen("my server", "ssh", hostname="machine.example.org", username="user"))

for i in range(10):
    script.builtin.file(state="touch", path=f"/tmp/test{i}")

How fast would that be, compared to Ansible?

$ time ansible-playbook test.yaml
[...]
real    0m15.232s
user    0m4.033s
sys 0m1.336s

$ time ./test_script

real    0m4.934s
user    0m0.547s
sys 0m0.049s

With a network round-trip for each single operation I'm already 3x faster than Ansible, and it can run on nspawn containers, too!

I always wanted to have a library of ansible modules useable in normal scripts, and I've always been angry with Ansible for not bundling their backend code in a generic library. Well, now there's the beginning of one!

Sweet! Next step, pipelining.

Enrico Zini: My gripes with Ansible [Planet Debian]

This is part of a series of posts on ideas for an ansible-like provisioning system, implemented in Transilience.

Musing about Ansible

I like infrastructure as code.

I like to be able to represent an entire system as text files in a git repositories, and to be able to use that to recreate the system, from my Virtual Private Server, to my print server and my stereo, to build machines, to other kind of systems I might end up setting up.

I like that the provisioning work I do on a machine can be self-documenting and replicable at will.

The good

For that I quite like Ansible, in principle: simple (in theory) YAML files describe a system in (reasonably) high-level steps, and it can be run on (almost) any machine that happens to have a simple Python interpreter installed.

I also like many of the modules provided with Ansible: they are convenient, platform-independent implementations of common provisioning steps. They'd be fantastic to have in a library that I could use in normal programs.

The bad

Unfortunately, Ansible is slow. Running the playbook on my VPS takes about 3 whole minutes even if I'm just changing a line in a configuration file.

This means that most of the time, instead of changing that line in the playbook and running it, to then figure out after 3 minutes that it was the wrong line, or I made a spelling mistake in the playbook, I end up logging into the server and editing in place.

That defeats the whole purpose, but that level of latency between iterations is just unacceptable to me.

The ugly

I also think that Ansible has outgrown its original design, and the supposedly declarative, idempotent YAML has become a full declarative scripting language in disguise, whose syntax is extremely awkward and verbose.

If I'm writing declarative descriptions, YAML is great. If I'm writing loops and conditionals, I want to write code, not templated YAML.

I also keep struggling trying to use Ansible to provision chroots and nspawn containers.

A personal experiment: Transilience

There's another thing I like in Ansible: it's written in Python, which is a language I'm comfortable with. Compared to other platforms, it's one that I'm more likely to be able to control beyond being a simple user.

What if I can port Ansible modules into a library of high-level provisioning functions, that I can just run via normal Python scripts?

What if I can find a way to execute those scripts remotely and not just locally?

I've started writing some prototype code, and the biggest problem is, of course, finding a name.

Ansible comes from Ursula K. Le Guin's Hainish Cycle novels, where it is a device that allows its users to communicate near-instantaneously over interstellar distances. Traveling, however, is still constrained by the speed of light.

Later in the same universe, the novels A Fisherman of the Inland Sea and The Shobies' Story, talk about experiments with instantaneous interstellar travel, as a science Ursula Le Guin called transilience:

Transilience: n. A leap across or from one thing to another [1913 Webster]

Transilience. I like everything about this name.

Now that the hardest problem is solved, the rest is just a simple matter of implementation details.

15:42

The ARM processor (Thumb-2), part 11: Atomic access and barriers [The Old New Thing]

On the ARM processor, atomic operations are implemented in terms of a load-locked/store-conditional pair of instructions.

    LDREX   Rd, [Rn, #imm8]     ; load word from [Rn, #imm8] and acquire exclusively
    STREX   Rd, Rm, [Rn, #imm8] ; store Rm to [Rn, #imm8] if exclusively held
                                ; Rd = 0 on success or 1 on failure

    ; also LDREXB, LDREXH, LDREXD
    ;      STREXB, STREXH, STREXD

    CLREX                       ; release exclusive lock

The LDREX instruction loads a word from the specified address and takes an exclusive lock on the memory. This exclusive lock is broken if any other processor writes to the same address, or if the lock is explicitly cleared. The granularity of the lock is permitted to be as coarse as 2KB.

The STREX instruction writes the value Rm to Rn provided the exclusive lock has not been lost. The Rd register is set to 0 if the write succeeded, or 1 if the write failed. The Rd register may not be the same register as Rm.

The STREX is permitted to early-out and return failure due to a lost lock before checking whether the memory at Rn is writable.

The LDREX and STREX instructions support only offset addressing with an unsigned 8-bit offset. (An offset of zero is assumed if none is provided.) No pre-indexing or post-indexing allowed.

There are also byte, word, and doubleword versions of this pair of instructions. For best results, use the STREX variant that matches the LDREX variant, and with the same address.

You can explicitly abandon a lock obtained by one of the LDREX instructions by issuing a CLREX instruction. This is used primarily in kernel mode to ensure that interrupts and context switches cause the lock to be lost: If the user-mode code is interrupted between the LDREX and the subsequent STREX, you want to make sure the STREX fails, rather than accidentally succeeding because it’s writing to an address that coincidentally matches a previous LDREX from the outgoing thread or interrupt.

The atomic memory access instructions require aligned memory. Relaxing alignment enforcement doesn’t help here. Not that you expect it to: How can the kernel emulate a misaligned lock?

The atomic memory operations are frequently coupled with synchronization primitives. The ARM processor has a rather weak memory model, so memory barriers are essential in proper multithreaded code.

    DMB     ish     ; data memory barrier
    DSB     ish     ; data synchronization barrier
    ISB     sy      ; instruction synchronization barrier

The data memory barrier ensures that all preceding writes are issued before any subsequent memory operations (including speculative memory access). In acquire/release terms, it is a full barrier. The instruction does not stall execution; it just tells the memory controller to preserve externally-visible ordering. This is probably the only barrier you will ever seen in user-mode code.

The data synchronization barrier is a data memory barrier, but with the additional behavior of stalling until all outstanding writes have completed. This is typically used during context switches.

The instruction synchronization barrier flushes instruction prefetch. This is typically used if you have generated new code, say by jitting it or paging it in from disk.

All of the barrier instructions take a parameter known as the sychronization domain. In practice, they will be the values I gave in the examples above.

A typical atomic sequence, complete with memory barriers, looks like this:

    dmb     ish             ; memory barrier

@@: ldrex   r2, [r0]        ; load r2 from [r0] and lock

    ; calculate new value - in this example, we increment
    adds    r2, r2, #1      ; increment it

    strex   r3, r2, [r0]    ; store if lock is still held
    cmp     r3, #0          ; did it succeed?
    bne     @B              ; N: try again

    dmb     ish             ; memory barrier

Finally, we have some instructions that provide hints to the processor about future memory usage:²

    PLD     [Rn, #imm]      ; preload data
    PLDW    [Rn, #imm]      ; preload data with intent to write
    PLI     [Rn, #imm]      ; preload instructions

Processors are not required to honor these instructions and may treat them as nop. (Pre-index and post-index are not supported, so you don’t have to worry about accidentally nop’ing out the write-back.) If the address being prefetched is not valid, the request is ignored.

Okay, enough about memory. Next time, we’ll look at control transfer instructions.

Bonus chatter: Classic ARM also contains two deprecated pseudo-atomic instructions:

    ; swap
    swp     Rt, Rt2, [Rn]   ; temp = [Rn]
                            ; [Rn] = Rt2
                            ; Rt = temp

    ; swap byte
    swpb    Rt, Rt2, [Rn]   ; temp = byte at [Rn]
                            ; byte at [Rn] = Rt2
                            ; Rt = temp (zero-extended)

These are pseudo-atomic instructions because the processor promises that it will not split the load and store, but only if no TLB eviction occurs, and it makes no promises about what other processors or devices may see.

These instructions are formally deprecated by ARM, and operating systems are permitted to disable them outright. Windows disables them, which is redundant because the instructions aren’t available in Thumb-2 mode anyway. I guess Windows wants to make extra sure you don’t use them.

¹ Even if alignment enforcement is relaxed, you will still get an alignment exception for misaligned doubleword access or any instruction that reads or writes multiple registers.

² Internally, these instructions reuse the encodings for loading partial values into pc, something you would never do in sane code. This is an example of how Thumb-2 disallows certain operations with pc and reuses the instruction encodings for other purposes.

Instruction Encoded as if
PLD  [...] LDRB  pc, [...]
PLDW [...] LDRH  pc, [...]
PLI  [...] LDRSB pc, [...]

The post The ARM processor (Thumb-2), part 11: Atomic access and barriers appeared first on The Old New Thing.

14:42

Link [Scripting News]

BTW, this is a non-smoking weblog since June 14, 2002.

14:07

1236: No Strings [Order of the Stick]

http://www.giantitp.com/comics/oots1236.html

14:00

Link [Scripting News]

The new Instant Outlines toolkit. Over the last few days I've been tweeting and blogging like a fool about why importing and exporting outlines from one app to another isn't enough. We can do much better. In the background I was working on a new toolkit for developers who want to be part of a network of outline producer and consumer apps based on OPML and web sockets. With source code of course, and a demo app to prove it works. All my outliners support the feature as well, it's the basis for sharing outlines. This is my vision for an open ecosystem based on outlining. It's a big part of what I dreamt of when I started UserLand in 1988.

Link [Scripting News]

Proof by induction. If a statement is true for 0, and if it being true for n implies it's true for n+1, then it's proven for all positive integers. When we let Dubya off for his sins, and when we didn't restructure the banking industry after the crash in 2008, we set the stage for it to happen again, only worse. And Trump isn't done with us yet. He's the weird case of a former president who hasn't relinquished power. He hasn't learned how to use it yet, but he'll get more effective in his new role.

12:35

CodeSOD: Experience is Integral [The Daily WTF]

Behind every code WTF is a process WTF. For example, Charles W was recently tasked with updating some file-handling code to match changes in the underlying file-format it parses. This is the C code...

11:42

Grrl Power #953 – Saucegate [Grrl Power]

If she’s not using them, I often forget about Sydney’s orbs until I’m placing them on the page, which I usually do near the end, and sometimes that results in the orbs being shoved off to the side while she eats. They’ll eventually drift back into place if she doesn’t give them a thought now and again.

Everyone at the base thinks Sydney and Frix are a cute couple, even if Fridney isn’t the best couple portmanteau. Some of that is just because of the ridiculous height difference. Cora has a type, and Frix is right at 7′ tall.

If Grrl Power was a different type of comic, I might spend some time on people saying that human/alien relationships are abominations and “against nature” all that jazz, but it’s just so… tired and predictable. I don’t mean predictable as a story trope, but predictable in a “humans suck and have always sucked and will probably continue to suck for the foreseeable future” kind of way.

Claiming things that a person doesn’t like are “against nature” always cracks me up, cause they’re usually doing so online, and we all know how natural computers are. Half of them are wearing eyeglasses, surely freshly picked from the eyeglasses trees. If you’re naked and yelling at a city from a hilltop forest and are riddled with parasites and ringworm and half your teeth are rotted out of your head and you die at 32, then knock your bad self out, otherwise, shut up about shit being natural. Humans are natural, therefore anything we can accomplish is by extension, natural. Unless you’re prepared to make the argument that beehives and beaver dams aren’t natural either.

Which is all my long winded way of saying yeah, there are people in the Grrl-verse that are against supers and humans dating supers and are against aliens and people dating them and allowing them on Earth (even though the only jobs aliens have stolen thus far is “tourist” – not including Dabbler, I suppose), there’s enough of that depressing stuff in the real world and the PR team does a pretty good job of keeping that sort of stuff out of the faces of Archon personnel.


Daniel Schinhofen’s third Luck’s Voice book came out last week. Breaking the Bank. I’m not a fan of wild west stuff at all, but this is wild west LitRPG (lite) with magic and elves and dwarves and the like, and Schinhofen has a pretty good track record IMO, so I gave it a shot. The audiobook for the first book in the series recently began recording as well, so keep an eye out for that as well once it gets through Amazon’s capricious submission gauntlet.

 

 

 


The new vote incentive is up! Welcome to Dabbler’s Damsel in Distress University.

Worried that you’ll end up in a deathtrap, precariously balancing a bomb with a mercury switch (or a canister full of tickle powder) while you struggle to escape? Then sign up now and you can practice your death (or tickle powder) defying escapes in the safety of the classroom!

Courses include Deathtrap 101-401, Tying up the Supervillain’s Henchwoman once you Escape Leaving her in the Very Same Precarious Deathtrap 101-201, Having Dinner with the Supervillian While he’s Holding the City Hostage 101-401, Holding Your Breath for Fun and Profit 101, Lockpicking 101-301, Ropework 101-406, Labyrinth Navigation 101, Blind Fighting, Oil Wrestling, Seducing the Guards 101-403, and many more!

As always, nude versions are up at Patreon.


Double res version will be posted over at Patreon. Feel free to contribute as much as you like!

10:00

Five useful questions [Seth's Blog]

They might be difficult to answer, but your project will benefit:

What’s the hard part? Which part of your work, if it suddenly got much better, would have the biggest impact on the outcome you seek?

How are you spending your time? If we took at look at your calendar, how much time is spent reacting or responding to incoming, how much is under your control, and how much is focused on the hard part?

What do you need to know? What are the skills that you don’t have that would make your work more effective?

What is the scary part? Which outcomes or interactions are you trying to avoid thinking about or interacting with? Why?

Is it worth it? After looking at your four answers to these questions, you might have a better idea of what it will take for your project to reach its potential. Does the outcome of the project–for those you serve and for you–justify what it will take to get it there?

08:42

Comic: A Thousand Scripts [Penny Arcade]

New Comic: A Thousand Scripts

08:28

Urgent: Fire and replace a USPS board member [Richard Stallman's Political Notes]

US citizens: call on Biden to fire and replace one of the USPS board members, to assure the firing of DeJoy.

Exacerbated effects of PTSD [Richard Stallman's Political Notes]

The effects of PTSD on US war veterans are exacerbated by the fact that the US doesn't want to be a society which cheerfully accepts killing. We want to be a society that fights wars only when just. Thus, veterans feel guilt about what they have done in war.

This makes a paradoxical contrast with the fact that the US fights so many avoidable wars that one can hardly keep track of them all. For most of these wars, there is no possibility of real victory, which means the US has to choose between indefinite prolongation and politically unacceptable voluntary defeat.

Perhaps the best way to reduce the harm done to Americans by PTSD is by learning to be less ready to fight a war. If we only fought when there was a reason to be proud afterward of having fought, perhaps we would fight fewer wars, fewer veterans would suffer moral injury, and those who did could be welcomed back and healed.

Other countries have similar problems. Samantha Crompvoets interviewed Australian veterans about war crimes, and stated conclusions that the Australian government would rather silence.

Right-wing trumpery [Richard Stallman's Political Notes]

Right-wing trumpery: running ads on Facebook for Green Party candidates in the hope of dividing Democrats, and pretending not to be who they really were.

Sri Lanka pollution ship disaster [Richard Stallman's Political Notes]

The Sri Lanka pollution ship disaster can be traced to the country's debts, which forced it to lease its ports, long term, to foreign investors that didn't care about safety.

Trade war [Richard Stallman's Political Notes]

The UK and the EU are headed for a trade war.

Bogus Johnson saw a collision coming, and pretended for political purposes that it would magically go away.

*How Tories changed their tune on Northern Ireland protocol.*

Discouraging smoking [Richard Stallman's Political Notes]

In Britain, smokers overall give more support to discouraging smoking than to protecting the right to smoke.

Tobacco is death for the smoker, as well as disgusting for everyone else. I wish I could magically help everyone quit. Nonetheless, I oppose prohibition of tobacco, on grounds of personal freedom.

Censorship on movies in Hong Kong [Richard Stallman's Political Notes]

China will impose political censorship on movies in Hong Kong.

If you are in Hong Kong, the only way to get copies of movies is through unofficial channels, sharing from people in countries which have only some of the oppressive systems that Hong Kong and China have.

Deals with informants in the Mafia [Richard Stallman's Political Notes]

Right-wing Italian prohibitions want to repeal the law that allows making deals with informants in the Mafia. They want to have their cake (long prison sentences) and eat it too (get confessions and convictions).

Mexican children sent back to danger [Richard Stallman's Political Notes]

*The Biden Administration Is Routinely Sending Mexican Children Back to Danger, Report Finds.*

US military is socialist [Richard Stallman's Political Notes]

The US military is the largest socialist entity on Earth. Strange how officers tend to despise socialism.

New workplace rules about Covid-19 [Richard Stallman's Political Notes]

The US Labor Department's new workplace rules about Covid-19 fail to require most workplaces to do anything to protect employees.

Greenhouse emissions reduction [Richard Stallman's Political Notes]

An enormous group of investment funds are calling on governments to speed up greenhouse emissions reduction.

Two notable exceptions are Blackrock and Vanguard, which deserve specific rejection by the public.

If these funds are serious about the matter, they could exert strong pressure on corporations as stockholders. Their lobbying power could be significant too.

Shortfall in funds [Richard Stallman's Political Notes]

The UK has received the suggestion to fill a shortfall in funds for universities from low-income graduates, by reducing the income threshold above which they must repay their student loans.

That sounds like the sort of soak-the-poor plan that Tories would love.

(satire) New chalet in the Swiss Alps [Richard Stallman's Political Notes]

(satire) *Pfizer Announces Breakthrough Medication That Will Treat Executives To New Chalet In Swiss Alps.*

Communication records of Democrats [Richard Stallman's Political Notes]

The would-be tyrant had the Department of Justice seize communications records of Democrats in Congress and their staff and relatives, as part of a leak investigation.

In this article, an official claims this was meant as punishment, not real investigation.

Chip factories in Taiwan [Richard Stallman's Political Notes]

Chip factories in Taiwan are having trouble getting enough water, as Taiwan suffers form a drought. Global heating will lead to more and worse droughts there.

Powerful countries should not allow their manufacture of any critical product to be concentrated in one region or a few regions. Such centralization leads to vulnerability — to various kinds of problems.

The way to reduce the vulnerability, to threats of many unrelated kinds, is to disperse production.

Anti-corruption crusaders [Richard Stallman's Political Notes]

Anti-corruption crusaders in Guatemala have been charged with crimes that are incredible.

I can't imagine that anything could motivate those influential people to intentionally found a political party and list a dead person as participant. Why not find a living supporter to list instead? I conclude the charges must have been fabricated.

Colombia's defense minister [Richard Stallman's Political Notes]

Colombia's former president Santos was previously the defense minister and thus ultimately responsible for the over 6,000 murders committed by soldiers during that time. He says that he took action to stop the practice when he determined it was happening. Nonetheless, he asks forgiveness.

Santos said that the pressure for the murders came from the then president, Alvaro Horrible (Uribe).

I think that those who killed, or encouraged or facilitated the killing, ought to be prosecuted for it.

Invasive species as pets [Richard Stallman's Political Notes]

An article about keeping an invasive species as pets (Australian possums in New Zealand) presents the usual emotional pleas, plus a peculiar nonsequitur about colonial history.

Perhaps it would be safe and reasonable to permit neutered possums as pets in New Zealand, provided they are not allowed outdoors. The article does not say whether the pet-owners interviewed do that.

One of them has a cat, too. Cats in New Zealand are a similar case. Even in the US, cats that are allowed outside devastate many species of birds. If you want to have a cat, please keep it indoors and protect wildlife.

Starving people in Tigray [Richard Stallman's Political Notes]

The UN official in charge of food aid accuses the Eritrian army of starving people in Tigray.

Mother Corona [Richard Stallman's Political Notes]

One Indian village is praying to a new goddess, "Mother Corona", to stop the disease.

That's where religion comes from.

G7 global tax system [Richard Stallman's Political Notes]

The G7's agreement on a global tax system for multinational companies will mostly increase tax revenue for their home countries. It won't do much good for poor countries they operate in.

Whistleblower Craig Murray [Richard Stallman's Political Notes]

*Whistleblower Craig Murray Sentenced To 8 Months In Prison Over His Reporting On Former Scottish First Minister's Trial.*

08:21

Open-Ended [Ctrl+Alt+Del Comic]

But don’t worry, eventually they’ll release the “HBO” DLC, where the you can watch a couple of idiots make up their own ending to the game.

The post Open-Ended appeared first on Ctrl+Alt+Del Comic.

07:56

François Marier: Self-hosting an Ikiwiki blog [Planet Debian]

8.5 years ago, I moved my blog to Ikiwiki and Branchable. It's now time for me to take the next step and host my blog on my own server. This is how I migrated from Branchable to my own Apache server.

Installing Ikiwiki dependencies

Here are all of the extra Debian packages I had to install on my server:

apt install ikiwiki ikiwiki-hosting-common gcc libauthen-passphrase-perl libcgi-formbuilder-perl libcrypt-sslauthen-passphrase-perl libcgi-formbuilder-perl libcrypt-ssleay-perl libjson-xs-perl librpc-xml-perl python-docutils libxml-feed-perl libsearch-xapian-perl libmailtools-perl highlight-common libsearch-xapian-perl xapian-omega
apt install --no-install-recommends ikiwiki-hosting-web libgravatar-url-perl libmail-sendmail-perl libcgi-session-perl
apt purge libnet-openid-consumer-perl

Then I enabled the CGI module in Apache:

a2enmod cgi

and un-commented the following in /etc/apache2/mods-available/mime.conf:

AddHandler cgi-script .cgi

Creating a separate user account

Since Ikiwiki needs to regenerate my blog whenever a new article is pushed to the git repo or a comment is accepted, I created a restricted user account for it:

adduser blog
adduser blog sshuser
chsh -s /usr/bin/git-shell blog

git setup

Thanks to Branchable storing blogs in git repositories, I was able to import my blog using a simple git clone in /home/blog (the srcdir):

git clone --bare git://feedingthecloud.branchable.com/ source.git

Note that the name of the directory (source.git) is important for the ikiwikihosting plugin to work.

Then I pulled the .setup file out of the setup branch in that repo and put it in /home/blog/.ikiwiki/FeedingTheCloud.setup. After that, I deleted the setup branch and the origin remote from that clone:

git branch -d setup
git remote rm origin

Following the recommended git configuration, I created a working directory (the repository) for the blog user to modify the blog as needed:

cd /home/blog/
git clone /home/blog/source.git FeedingTheCloud

I added my own ssh public key to /home/blog/.ssh/authorized_keys so that I could push to the srcdir from my laptop.

Finaly, I generated a new ssh key without a passphrase:

ssh-keygen -t ed25519

and added it as deploy key to the GitHub repo which acts as a read-only mirror of my blog.

Ikiwiki config

While I started with the Branchable setup file, I changed the following things in it:

adminemail: webmaster@fmarier.org
srcdir: /home/blog/FeedingTheCloud
destdir: /var/www/blog
url: https://feeding.cloud.geek.nz
cgiurl: https://feeding.cloud.geek.nz/blog.cgi
cgi_wrapper: /var/www/blog/blog.cgi
cgi_wrappermode: 675
add_plugins:
- goodstuff
- lockedit
- comments
- blogspam
- sidebar
- attachment
- favicon
- format
- highlight
- search
- theme
- moderatedcomments
- flattr
- calendar
- headinganchors
- notifyemail
- anonok
- autoindex
- date
- relativedate
- htmlbalance
- pagestats
- sortnaturally
- ikiwikihosting
- gitpush
- emailauth
disable_plugins:
- brokenlinks
- fortune
- more
- openid
- orphans
- passwordauth
- progress
- recentchanges
- repolist
- toggle
- txt
sslcookie: 1
cookiejar:
  file: /home/blog/.ikiwiki/cookies
useragent: ikiwiki
git_wrapper: /home/blog/source.git/hooks/post-update
urlalias:
- http://feeds.cloud.geek.nz/
- http://www.feeding.cloud.geek.nz/
owner: francois@fmarier.org
hostname: feeding.cloud.geek.nz
emailauth_sender: login@fmarier.org
allowed_attachments: admin()

Then I created the destdir:

mkdir /var/www/blog
chown blog:blog /var/www/blog

and generated the initial copy of the blog as the blog user:

ikiwiki --setup .ikiwiki/FeedingTheCloud.setup --wrappers --rebuild

One thing that failed to generate properly was the tag cloug (from the pagestats plugin). I have not been able to figure out why it fails to generate any output when run this way, but if I push to the repo and let the git hook handle the rebuilding of the wiki, the tag cloud is generated correctly. Consequently, fixing this is not high on my list of priorities, but if you happen to know what the problem is, please reach out.

Apache config

Here's the Apache config I put in /etc/apache2/sites-available/blog.conf:

<VirtualHost *:443>
    ServerName feeding.cloud.geek.nz

    SSLEngine On
    SSLCertificateFile /etc/letsencrypt/live/feeding.cloud.geek.nz/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/feeding.cloud.geek.nz/privkey.pem

    Header set Strict-Transport-Security: "max-age=63072000; includeSubDomains; preload"

    Include /etc/fmarier-org/blog-common
</VirtualHost>

<VirtualHost *:443>
    ServerName www.feeding.cloud.geek.nz
    ServerAlias feeds.cloud.geek.nz

    SSLEngine On
    SSLCertificateFile /etc/letsencrypt/live/feeding.cloud.geek.nz/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/feeding.cloud.geek.nz/privkey.pem

    Redirect permanent / https://feeding.cloud.geek.nz/
</VirtualHost>

<VirtualHost *:80>
    ServerName feeding.cloud.geek.nz
    ServerAlias www.feeding.cloud.geek.nz
    ServerAlias feeds.cloud.geek.nz

    Redirect permanent / https://feeding.cloud.geek.nz/
</VirtualHost>

and the common config I put in /etc/fmarier-org/blog-common:

ServerAdmin webmaster@fmarier.org

DocumentRoot /var/www/blog

LogLevel core:info
CustomLog ${APACHE_LOG_DIR}/blog-access.log combined
ErrorLog ${APACHE_LOG_DIR}/blog-error.log

AddType application/rss+xml .rss

<Location /blog.cgi>
        Options +ExecCGI
</Location>

before enabling all of this using:

a2ensite blog
apache2ctl configtest
systemctl restart apache2.service

The feeds.cloud.geek.nz domain used to be pointing to Feedburner and so I need to maintain it in order to avoid breaking RSS feeds from folks who added my blog to their reader a long time ago.

Server-side improvements

Since I'm now in control of the server configuration, I was able to make several improvements to how my blog is served.

First of all, I enabled the HTTP/2 and Brotli modules:

a2enmod http2
a2enmod brotli

and enabled Brotli compression by putting the following in /etc/apache2/conf-available/francois.conf:

<IfModule mod_brotli.c>
    AddOutputFilterByType BROTLI_COMPRESS text/html text/plain text/xml text/css text/javascript application/javascript
    BrotliCompressionQuality 4
</IfModule>

Next, I made my blog available as a Tor onion service by putting the following in /etc/apache2/sites-available/blog.conf:

<VirtualHost *:443>
    ServerName feeding.cloud.geek.nz
    ServerAlias xfdug5vmfi6oh42fp6ahhrqdjcf7ysqat6fkp5dhvde4d7vlkqixrsad.onion

    Header set Onion-Location "http://xfdug5vmfi6oh42fp6ahhrqdjcf7ysqat6fkp5dhvde4d7vlkqixrsad.onion%{REQUEST_URI}s"
    Header set alt-svc 'h2="xfdug5vmfi6oh42fp6ahhrqdjcf7ysqat6fkp5dhvde4d7vlkqixrsad.onion:443"; ma=315360000; persist=1'
    ... 
<VirtualHost *:80>
    ServerName xfdug5vmfi6oh42fp6ahhrqdjcf7ysqat6fkp5dhvde4d7vlkqixrsad.onion
    Include /etc/fmarier-org/blog-common
</VirtualHost>

Then I followed the Mozilla Observatory recommendations and enabled the following security headers:

Header set Content-Security-Policy: "default-src 'none'; report-uri https://fmarier.report-uri.com/r/d/csp/enforce ; style-src 'self' 'unsafe-inline' ; img-src 'self' https://seccdn.libravatar.org/ ; script-src https://feeding.cloud.geek.nz/ikiwiki/ https://xfdug5vmfi6oh42fp6ahhrqdjcf7ysqat6fkp5dhvde4d7vlkqixrsad.onion/ikiwiki/ http://xfdug5vmfi6oh42fp6ahhrqdjcf7ysqat6fkp5dhvde4d7vlkqixrsad.onion/ikiwiki/ 'unsafe-inline' 'sha256-pA8FbKo4pYLWPDH2YMPqcPMBzbjH/RYj0HlNAHYoYT0=' 'sha256-Kn5E/7OLXYSq+EKMhEBGJMyU6bREA9E8Av9FjqbpGKk=' 'sha256-/BTNlczeBxXOoPvhwvE1ftmxwg9z+WIBJtpk3qe7Pqo=' ; base-uri 'self'; form-action 'self' ; frame-ancestors 'self'"
Header set X-Frame-Options: "SAMEORIGIN"
Header set Referrer-Policy: "same-origin"
Header set X-Content-Type-Options: "nosniff"

Note that the Mozilla Observatory is mistakenly identifying HTTP onion services as insecure, so you can ignore that failure.

I also used the Mozilla TLS config generator to improve the TLS config for my server.

Then I added security.txt and gpc.json to the root of my git repo and then added the following aliases to put these files in the right place:

Alias /.well-known/gpc.json /var/www/blog/gpc.json
Alias /.well-known/security.txt /var/www/blog/security.txt

I also followed these instructions to create a sitemap for my blog with the following alias:

Alias /sitemap.xml /var/www/blog/sitemap/index.rss

Finally, I simplified a few error pages to save bandwidth:

ErrorDocument 301 " "
ErrorDocument 302 " "
ErrorDocument 404 "Not Found"

Monitoring 404s

Another advantage of running my own web server is that I can monitor the 404s easily using logcheck by putting the following in /etc/logcheck/logcheck.logfiles:

/var/log/apache2/blog-error.log 

Based on that, I added a few redirects to point bots and users to the location of my RSS feed:

Redirect permanent /atom /index.atom
Redirect permanent /comments.rss /comments/index.rss
Redirect permanent /comments.atom /comments/index.atom
Redirect permanent /FeedingTheCloud /index.rss
Redirect permanent /feed /index.rss
Redirect permanent /feed/ /index.rss
Redirect permanent /feeds/posts/default /index.rss
Redirect permanent /rss /index.rss
Redirect permanent /rss/ /index.rss

and to tell them to stop trying to fetch obsolete resources:

Redirect gone /~ff/FeedingTheCloud
Redirect gone /gittip_button.png
Redirect gone /ikiwiki.cgi

I also used these 404s to discover a few old Feedburner URLs that I could redirect to the right place using archive.org:

Redirect permanent /feeds/1572545745827565861/comments/default /posts/watch-all-of-your-logs-using-monkeytail/comments.atom
Redirect permanent /feeds/1582328597404141220/comments/default /posts/news-feeds-rssatom-for-mythtvorg-and/comments.atom
...
Redirect permanent /feeds/8490436852808833136/comments/default /posts/recovering-lost-git-commits/comments.atom
Redirect permanent /feeds/963415010433858516/comments/default /posts/debugging-openwrt-routers-by-shipping/comments.atom

I also put the following robots.txt in the git repo in order to stop a bunch of authentication errors coming from crawlers:

User-agent: *
Disallow: /blog.cgi
Disallow: /ikiwiki.cgi

Future improvements

There are a few things I'd like to improve on my current setup.

The first one is to remove the iwikihosting and gitpush plugins and replace them with a small script which would simply git push to the read-only GitHub mirror. Then I could uninstall the ikiwiki-hosting-common and ikiwiki-hosting-web since that's all I use them for.

Next, I would like to have proper support for signed git pushes. At the moment, I have the following in /home/blog/source.git/config:

[receive]
    advertisePushOptions = true
    certNonceSeed = "(random string)"

but I'd like to also reject unsigned pushes.

While my blog now has a CSP policy which doesn't rely on unsafe-inline for scripts, it does rely on unsafe-inline for stylesheets. I tried to remove this but the actual calls to allow seemed to be located deep within jQuery and so I gave up. Patches for this would be very welcome of course.

Finally, I'd like to figure out a good way to deal with articles which don't currently have comments. At the moment, if you try to subscribe to their comment feed, it returns a 404. For example:

[Sun Jun 06 17:43:12.336350 2021] [core:info] [pid 30591:tid 140253834704640] [client 66.249.66.70:57381] AH00128: File does not exist: /var/www/blog/posts/using-iptables-with-network-manager/comments.atom

This is obviously not ideal since many feed readers will refuse to add a feed which is currently not found even though it could become real in the future. If you know of a way to fix this, please let me know.

06:28

The Vole Story – DORK TOWER 14.06.21 [Dork Tower]

There’s a DORK TOWER Patreon! Dork Tower is 100% reader-funded, and updated Mondays, Wednesdays and Fridays, thanks its generous Patreon supporters. The next goal is four strips a week! Enlist in the Army of Dorkness today, and help us reach that! We have a ton of fun! Also: Igor Bars!

06:21

1513 [Looking For Group]

The post 1513 appeared first on Looking For Group.

05:35

Girl Genius for Monday, June 14, 2021 [Girl Genius]

The Girl Genius comic for Monday, June 14, 2021 has been posted.

04:49

Tonight’s comic isn’t quite ready to date again 🤖 [Diesel Sweeties webcomic by rstevens]

this is a diesel sweeties comic strip

Remember kids: it's not a hookup unless they have an ethernet jack.

04:14

The Rent’s Too Damned High [Cory Doctorow's craphound.com]

This week on my podcast, my latest Medium column, The Rent’s Too Damned High, about the long con of convincing Americans that they will grow prosperous through housing wealth, not labor rights.

MP3

02:14

E-Tail [QC RSS]

A lot of y'all asked for it, so here!!!!!!! Available for preorder now on my Topatoco store!

00:49

Kernel prepatch 5.13-rc6 [LWN.net]

The 5.13-rc6 kernel prepatch is out for testing. "Nothing particularly special to say about this - rc6 is certainly smaller than rc5 was, so we're moving in the right direction".

Sunday, 13 June

22:35

Vincent Fourmond: Solution for QSoas quiz #2: averaging several Y values for the same X value [Planet Debian]

This post describes two similar solutions to the Quiz #2, using the data files found there. The two solutions described here rely on split-on-values. The first solution is the one that came naturally to me, and is by far the most general and extensible, but the second one is shorter, and doesn't require external script files.

Solution #1

The key to both solution is to separate the original data into a series of datasets that only contain data at a fixed value of x (which corresponds here to a fixed pH), and then process each dataset one by one to extract the average and standard deviation. This first step is done thus:
QSoas> load kcat-vs-ph.dat
QSoas> split-on-values pH x /flags=data
After these commands, the stacks contains a series of datasets bearing the data flag, that each contain a single column of data, as can be seen from the beginnings of a show-stack command:
QSoas> k
Normal stack:
         F  C   Rows    Segs    Name    
#0      (*) 1   43      1       'kcat-vs-ph_subset_22.dat'
#1      (*) 1   44      1       'kcat-vs-ph_subset_21.dat'
#2      (*) 1   43      1       'kcat-vs-ph_subset_20.dat'
...
Each of these datasets have a meta-data named pH whose value is the original x value from kcat-vs-ph.dat. Now, the idea is to run a stats command on the resulting datasets, extracting the average value of x and its standard deviation, together with the value of the meta pH. The most natural and general way to do this is to use run-for-datasets, using the following script file (named process-one.cmds):
stats /meta=pH /output=true /stats=x_average,x_stddev
So the command looks like:
QSoas> run-for-datasets process-one.cmds flagged:data
This command produces an output file containing, for each flagged dataset, a line containing x_average, x_stddev, and pH. Then, it is just a matter of loading the output file and shuffling the columns in the right order to get the data in the form asked. Overall, this looks like this:
l kcat-vs-ph.dat
split-on-values pH x /flags=data
output result.dat /overwrite=true
run-for-datasets process-one.cmds flagged:data
l result.dat
apply-formula tmp=y2;y2=y;y=x;x=tmp
dataset-options /yerrors=y2
The slight improvement over what is described above is the use of the output command to write the output to a dedicated file (here result.dat), instead of out.dat and ensuring it is overwritten, so that no data remains from previous runs.

Solution #2

The second solution is almost the same as the first one, with two improvements:
  • the stats command can work with datasets other than the current one, by supplying them to the /buffers= option, so that it is not necessary to use run-for-datasets;
  • the use of the output file can by replaced by the use of the accumulator.
This yields the following, smaller, solution:
l kcat-vs-ph.dat
split-on-values pH x /flags=data
stats /meta=pH /accumulate=* /stats=x_average,x_stddev /buffers=flagged:data
pop
apply-formula tmp=y2;y2=y;y=x;x=tmp
dataset-options /yerrors=y2


About QSoas

QSoas is a powerful open source data analysis program that focuses on flexibility and powerful fitting capacities. It is released under the GNU General Public License. It is described in Fourmond, Anal. Chem., 2016, 88 (10), pp 5050–5052. Current version is 3.0. You can download its source code there (or clone from the GitHub repository) and compile it yourself, or buy precompiled versions for MacOS and Windows there.

22:14

Is JSON better than XML? No. [Scripting News]

I've been all around this one. I can and often do save both XML and JSON versions of RSS and OPML. Truth is almost no one uses the JSON versions. The reason is simple. If you already have support for the XML version, what's gained by adding support for JSON? And everyone has to support the XML version, because HTML, RSS and OPML are all out there only in the XML format, for all practical purposes.

Now, to the supposed advantages of JSON over XML.

  • JSON is supposedly more readable than XML, but if you look at what people actually put on the wire, it's not true. There are no rules about whitespace, tabs, newlines, spaces in either format. Developers save files without including whitespace, thinking this makes their apps more efficient, when any gain has to be miniscule, esp with compression. It's bad engineering not to try to make your data files readable, imho, but it's also the default practice.
  • JSON is supposed to be easier to process, but that also is not true. First you have to support the XML versions because that's what's out there. So even if it were monumentally more difficult, you can't avoid supporting XML formatted data in your code. Adding a second version makes your code more complex not simpler.
  • If your development runtime doesn't have a package to serialize and deserialize XML as easily as JSON.parse and JSON.stringify, then do a good deed for your fellow developers, write it and release it as open source. That's a simpler and more ecologically sound way of alleviating any angst about XML.
  • The old famous programming adage applies -- if it ain't broke don't fix it. XML works. It doesn't matter to users if you use XML or JSON. If you want to make the world a friendlier place, use indentation and newlines in your data. Either XML or JSON can be readable if you put in a little effort.

21:56

Off the Musical Deep End, Part II: The Unpackening [Whatever]

John Scalzi

As a follow-up to the post from a couple of weeks ago, I now have the music room largely set up; there are a few more things I need to do and get (some acoustical tile; an actual chair), but they’re relatively minor things. I’m ready to fall all the way down the rabbit hole with this stuff now. If I can’t make music with what I have at this point, the problem is me, not what I have to work with.

What you’re not seeing here is the actual mountain of boxes and shipping material much of this stuff came in, so much of it that I think I need to donate to the Arbor Day Society to make up for all the cardboard I caused to be used.

(Oh, and: I did end up getting a Mac after all; a new Mac Mini. For two reasons: One, The Dell is a capable machine but like a lot of ultraportables doesn’t have a lot of physical connectivity. The Mac Mini does and it turns out that’s actually useful with a room full of physical equipment. Two, at the end of the day there’s more and better music creation stuff in the Apple ecosystem, and that’s what I’ll be using this particular computer for. Also, three, Krissy was all, “I know you want one, just get the damn thing,” and who am I to argue with Krissy.)

Again, my plan is when I’m not in my office, writing words, I’m down here in the basement, writing music. I’m not giving up the day job, to be sure. But this isn’t meant to be a side hustle. It’s just meant to be enjoyable for me. And I’m having fun already, so that’s good.

— JS

17:21

Pluralistic: 13 Jun 2021 [Pluralistic: Daily links from Cory Doctorow]


Today's links



Take2/Polar Lights Hauned Manor 'Play It Again, Tom' model-kit box.

A stroll through Magnolia Park (permalink)

One of the weird ironies of living in the US and having family, friends and colleagues abroad is the vast, iniquitous gap in vaccine availability based on where you live, and, more particularly, whether you live in a poor country or a rich one.

Vaccine Apartheid is a global terror and horror, but that's not the "ironic" part. That would be the American vaccine deniers who have effectively killed the dream of herd immunity, and taken anti-vax from a threat to public health to a threat to civilization itself.

The way this manifests is often quirky and personal – like the news that some of my beloved cousins in Canada and the US have become anti-vax, anti-mask conspiracists, losing themselves in the Qanon cult.

They're never far from my thoughts, but doubly so yesterday. You see, here in LA, we have high levels of vaccination and a general lifting of restrictions that – in contrast to the premature "re-openings" elsewhere that led to lethal outbreaks – feel prudent and safe.

That's given my neighborhood – Burbank's Magnolia Park – a new vitality. The centerpiece of the neighborhood is a couple miles' worth of pedestrian friendly, retail, dominated by independent and idiosyncratic retailers that draw people from all over the city.

Many of these did not survive the pandemic, but a heartening number of them held on, and it's great to see crowds out there on a Saturday. Yesterday, I rode my bike up to one end of the strip, outside Porto's, the regionally famous Cuban sandwich shop, locked up and strolled.

Magnolia Park's retail is dominated by vintage clothes and memorabilia stores, a legacy of our proximity to the studios (Disney, Warner and Universal are all a few minutes' drive), which created demand for wardrobe and set pieces, and a supply of post-shoot surplus items.

It's also got some great restaurants, like The New Deal. Unfortunately, thanks to Burbank's antiquated blue laws, almost no one has a real liquor license (wine and beer licenses are easy to get, but spirits licenses are all but impossible).

The sole exception on the strip is…unfortunate. Tinhorn Flats (AKA "Tinfoil Hats") is a fake saloon with a nice back garden that had one of those rare liquor license, and paired it with mediocre bar food. The best thing about it is its fantastic neon sign.

Tinhorn Flats, ringed with chainlink, doors barricaded with plywood and sandbags.

The worst thing about is that it's owned by mask-denying, covid-denying far right Trumpian conspiracists who defied public health orders, flooded their social media with culture war bullshit, and became a rallying point for every Bircher, Klansman and Qanon in the Valley.

Telephone pole flier for the weekly Tinhorn Flats Qanon rally.

I do mean "rallying point." As Tinhorn Flats waged war – installing generators after its power was cut, removing the boards over the door, etc – it hosted weekly Sat gatherings of unmasked, unhinged conspiracists waving American flags and signs decrying "Hollywood pedos."

They're still out there, every Saturday. If you're one of the many people who comes to our great family owned grocery Handy Market (whose neon is better than Tinhorn's!) for their weekly Saturday parking-lot BBQ, you've seen 'em, screaming about frazzledrip and "small business."

They were there yesterday, between my stops at The Mystic Museum and Halloween Town, two of our three goth superstores (the third being Dark Delicacies) – Burbank will costume you, sell you an articulated bat skeleton and fill your bookshelves.

Then you can tour the museum-grade replica of the horror section at a 1980s video store:

https://www.themysticmuseum.com/slashback

and buy merchandise from a wholly hypothetical slasher summer-camp:

https://beardedladysmysticmuseum.square.site/#MJosnZ

It's such an odd juxtaposition, to be walking around a neighborhood that is making a brave recovery from the lockdown, stopping in at these improbable, scrappy shops, and then walk past these superspreaders screaming in front of the chainlink-surrounded derelict bar.

Polar Lights Haunted Manor model kit boxes.

But my first fully vaccinated Saturday stroll down Magnolia was rescued by a discovery at Halloween Town: the discovery of Round2's "Haunted Manor" model kits, cheeky remakes of the classic "Zap/Action" MPC Haunted Mansion kits of the 1970s.

https://www.round2corp.com/?s=HAUNTED+MANOR&amp;post_type=product&amp;type_aws=true

MPC Zap/Action 1970s Haunted Mansion model kit boxes.

The original models were from the high-water mark of Haunted Mansion merch, the era of the UV-paint-doped "changing portrait" cards, the magnificent board-game, and Randotti skulls, models and plaques.

http://www.hauntedmansion.com/spgm/index.php?spgmGal=Vintage_Collectibles&amp;spgmPic=2#spgmPicture

They ingeniously incorporated rubber bands into their interiors to create pop-up effects, like a corpse that popped out a grave, causing the poor grave-digger to spin about. Between the kinetics, glow-in-the-dark plastic, and a good paint job, these were just fantastic.

MPC 'Play It Again, Sam' model kit comic book ad.

Even if you never owned one of these kits, if you read comics in the 70s and early 80s, you can't have missed their distinctive, brilliantly conceived full-page comics ads. Small wonder that these kits sell for stupid money in the secondary market.

The Take2 models (sold under the Polar Lights mark) are not quite replicas of the MPC models (presumably they couldn't get a license), but they're fabulous reinterpretations of the vintage designs and I love the renaming (i.e. "Play It Again, Sam" becomes "Play It Again, Tom").

Vintage comic-book ad for Pirates of the Caribbean MPC Zap/Action model kits.

Alas, I couldn't find any sign of a Polar Lights remake of the MPC Zap/Action Pirates of the Caribbean models (whose ads were even better!).

https://pirates.fandom.com/wiki/Pirates_of_the_Caribbean_model_kits

After all that, I confess I didn't buy the kits (though I may go back today and rectify that). My daily work-load is so high that I'm lucky if I can manage to carve out half an hour every couple days to read a book, let alone put together and paint a model.

But of all the aspirational hobbies I'm wishing I was engaging in, assembling these models tops the list. Building a "Grave Robber's Demise" kit wouldn't quite be a "nature is healing" moment, but I know it would give me joy.

In the meantime, I hope you get vaccinated, too – and if you're ever in Burbank, be sure to patronize our wonderful indie stores (and don't miss Iliad Bookshop, one of the great used bookstores of the region!).



This day in history (permalink)

#20yrsago Nerve interviews Samuel R Delany https://web.archive.org/web/20010802205617/https://www.nerve.com/Dispatches/Westerfeld/SpaceCowboy/

#20yrsago Disney workers' union wins right to wear your own underwear under full-body costumes https://web.archive.org/web/20020228125231/http://www.sfgate.com/cgi-bin/article.cgi?file=/news/archive/2001/06/07/state1339EDT0171.DTL

#10yrsago A dog with persistence-of-vision LEDs in her shirt writes my novel Makers in the park at night https://web.archive.org/web/20110618011346/http://i.document.m05.de/?p=970

#10yrsago My head is a 3D scan https://www.makerbot.com/stories/news/new-york-notables-party-this-thursday-june-30th/

#5yrsago Microsoft will buy Linkedin for $26.2B https://arstechnica.com/information-technology/2016/06/microsoft-will-acquire-linkedin-for-18-5b/

#5yrsago Rio: your quadrennial reminder that the Olympics colonize host-states with Orwellian surveillance and human rights abuses https://www.vice.com/en/article/wnxgpw/the-olympics-are-turning-rio-into-a-military-state

#5yrsago China’s online astroturf is mostly produced by government workers as “extra duty” https://gking.harvard.edu/files/gking/files/50c.pdf?m=1464790150

#1yrago Facebook endorses Terra Nullius https://pluralistic.net/2020/06/13/robopinkertons/#filternet

#1yrago Secrets of a seventeen year old scraper https://pluralistic.net/2020/06/13/robopinkertons/#Avi-Schiffmann

#1yrago Facebook Workplace can block employees from discussing unionization https://pluralistic.net/2020/06/13/robopinkertons/#robopinkerton



Colophon (permalink)

Today's top sources:

Currently writing:

  • Spill, a Little Brother short story about pipeline protests. Friday's progress: 264 words (5483 words total).
  • A Little Brother short story about remote invigilation. PLANNING

  • A nonfiction book about excessive buyer-power in the arts, co-written with Rebecca Giblin, "The Shakedown." FINAL EDITS

  • A post-GND utopian novel, "The Lost Cause." FINISHED

  • A cyberpunk noir thriller novel, "Red Team Blues." FINISHED

Currently reading: Analogia by George Dyson.

Latest podcast: How To Destroy Surveillance Capitalism (Part 06) https://craphound.com/nonficbooks/destroy/2021/05/10/how-to-destroy-surveillance-capitalism-part-06/
Upcoming appearances:

Recent appearances:

Latest book:

Upcoming books:

  • The Shakedown, with Rebecca Giblin, nonfiction/business/politics, Beacon Press 2022

This work licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.


How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/web/accounts/303320

Medium (no ads, paywalled):

https://doctorow.medium.com/

(Latest Medium column: "The Rent’s Too Damned High," about the long con of convincing Americans that they will grow prosperous through housing wealth, not labor rights https://doctorow.medium.com/the-rents-too-damned-high-520f958d5ec5)

Twitter (mass-scale, unrestricted, third-party surveillance and advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla

16:14

Link [Scripting News]

This is important. In the conversations I've had recently with users and devs in the nascent Tools For Thought community, people talk in terms of importing and exporting outlines, but this emphatically is not good enough in 2021. We need a live connection. So when you hook your outline up to a rendering server, the connection persists. When you make a change, the worker app on the server has the changes and is ready to re-render, without you doing anything. I can't manually import a file every time there's a change. Web sockets are built into all browsers, it's simple and standardized tech. Let's use it. There's at least one public storage system, GitHub, that supports real time notification. Again, let's use it. Instead of going back to the beginning each time a new generation comes along, let's start with the work and knowhow of previous generations. I'm here to help.

Link [Scripting News]

Long ago, when RSS was starting to boom, I was often surprised when a new product came out and I didn't hear about it until it was announced publicly. The one I remember best was FeedBurner. You'd think that they would want my endorsement, and to give them a chance to answer technical questions before the press started asking them. Maybe they had a developer story? Or maybe I could have helped with a design decision? At that time Scripting News was pretty well-read in the developer community. It always felt like they must have been hiding something, but if they were I never found it. Yes FeedBurner centralized a technology that was good because it was decentralized, but I don't think my saying that would have hurt them. What was even more strange is I had friends at the company, people from the blogging world and from other RSS devs.

13:56

Link [Scripting News]

GitHub is an incredible free to use resource, reliable, has an excellent API and through webhooks is even real time. I think it could be a strong basis for interop between various outline generators and consumers.

10:07

Being the Queen sure has its ups and downs… [David Mitchell | The Guardian]

The media reaction to students voting to remove a portrait of the Queen has been cynically exploited by the Tories

Some good news at last! The middle common room of Magdalen College, Oxford has voted to remove its portrait of the Queen because of her association with colonialism. Don’t you think that’s really great news? There certainly seemed to be a consensus across the political spectrum that it was.

To be clear, I’m not saying there was a consensus that it was right to remove the picture. Far from it. That wouldn’t have been good news – that would have been exceptionally boring news. Everyone agreeing isn’t entertaining. This was great news because of the hysterical divergence of opinion about it.

The 'woke' and the patriotic hold a duopoly on self-righteousness

Continue reading...

10:00

False equivalencies [Seth's Blog]

It’s a pointless form of argument.

“This scientist made a careless error in their paper, therefore we need to excuse a con artist who falsified an entire career.”

Or, “that restaurant served fish that got someone sick, therefore, there’s no reason for there to be a health inspection at my restaurant or any other one for that matter.”

Or, “there was a typo in this book from a major publisher, so I’m not going to bother with an editor at all.”

The open-minded respond by trying to defend the original error or the intent behind it. But that simply amplifies the false equivalency argument and leads to a no-standards race to the bottom.

The false equivalency itself is the problem, not the unexpected error.

Perfect is a trap.

07:07

Urgent: Millionaires surtax [Richard Stallman's Political Notes]

US citizens: phone your congresscritter to support the "millionaires surtax", which would establish higher tax brackets for individuals with incomes over a million dollars a year.

The Capitol Switchboard number is 202-224-3121.

If you call, please spread the word!

Urgent: For the People Act [Richard Stallman's Political Notes]

US citizens: remind Senator Manchin that the For the People Act has overwhelming bipartisan support.

Greek labor bill [Richard Stallman's Political Notes]

Many Greek workers are on strike against a labor "flexibility" bill that would weaken their position in some ways against the demands of employers.

Biggest obstacle to ending Covid-19 [Richard Stallman's Political Notes]

Amnesty International: *The self-interest of G7 countries is the biggest obstacle to ending the Covid-19 pandemic, a group of campaigning organizations said today.*

To vaccinate the world fast requires building a lot more vaccine factories, fast. This requires eliminating the patent obstacle and making the vaccine companies teach others how to make those vaccines.

Under the present circumstances, with the way Big Pharma has corrupted the field of pharmaceuticals, patents on medicines to a lot of harm and very little good. (Patents in other fields don't do as much harm, but they still do very little good.) Any blow against the patent system will be a good thing.

However, building the additional vaccine factories may not be necessary. If China will finish vaccinating the world by early 2022, we might not have any way to speed that up by much.

Call for corporations to quit US Chamber of Commerce [Richard Stallman's Political Notes]

Student groups call on corporations to quit the US Chamber of Commerce unless it stops lobbying against climate defense.

Right-wing accusations of "corporate-run communism" [Richard Stallman's Political Notes]

Right-wing campaign groups are saying that CEOs are engaging in "corporate-run communism" when corporations state their opposition to voter suppression and election sabotage.

They know that right-wing lunatics today will accept any lie that supports their side.

Fossil fuel exit strategy [Richard Stallman's Political Notes]

The Fossil Fuel Exit Strategy presents a path to low greenhouse gas emissions that is feasible with no new technology.

Law to divest from fossil fuel [Richard Stallman's Political Notes]

Maine has passed a law to divest from fossil fuel assets.

Every government on Earth must do likewise.

(satire) Children's museum [Richard Stallman's Political Notes]

(satire) *Children’s Museum Docent Reminds Guests Not To Touch The Kids.*

Shell greenhouse emissions [Richard Stallman's Political Notes]

A Dutch court ruled that Shell must reduce its greenhouse emissions. A company executive makes fallacious arguments that this is unfair.

Wealth tax on Billionaires [Richard Stallman's Political Notes]

*Wealth Tax on World's Billionaires Would Raise $345 Billion a Year.*

(satire) Basic dignity to incentivize new hires [Richard Stallman's Political Notes]

(satire) *Desperate Employer Offers Basic Dignity To Incentivize New Hires.*

Danger of extracting fossil fuels [Richard Stallman's Political Notes]

A growing problem: resistance to fossil fuel infrastructure now frequently downplays the main danger of extracting and transporting fossil fuels — that they might leak methane into the air, or be burnt and release CO2. This example does not trouble to mention it at all.

What's with these people? I get the impression that they are more concerned with who owns certain land or waterways than with whether civilization survives.

Ironically, the same can be said of the companies that want to extract the fossil fuels.

The reason not to build the Line 3 pipeline applies to every other new oil or gas pipeline, every new refinery, and every new well or mine: there is >no room in the world's carbon budget for any more fossil fuel infrastructure.

Kevin Strickland murder conviction [Richard Stallman's Political Notes]

Kevin Strickland was convicted of murder in 1979. Since then, everyone involved in his case agrees he was falsely convicted, even the prosecutors. The governor of Missouri is grasping at straws to excuse keeping Strickland in prison anyway.

These strict "law and order" fanatics believe that it should be unthinkable to question a court's decision to punish someone. Unless that person is trying to destabilize our democracy for right-wing ideology.

Thugs charged for pepper spraying protesters [Richard Stallman's Political Notes]

Columbus, Ohio, thugs have been charged with crimes for spraying pepper spray on peaceful protesters who were violating no law or order, and for covering up lies.

A Louisville thug faces criminal charges for nonfatal brutality: hitting a suspect on the head with a stick while the latter was kneeling with his hands up.

No matter what that person was arrested for, or what he had actually done, nothing can justify hitting him under those circumstances.

Prosecuting thugs for unjustified violence even when it is nonfatal is crucial for teaching them to practice self-control as required for a police officer.

Gun manufacturers liability [Richard Stallman's Political Notes]

New York State has made gun manufacturers legally liable for damages caused by the manner of marketing or selling guns.

I want those activities to be better regulated, but I am concerned that this law may be unfair on account of vagueness. If those companies are to be required to obey new rules, the rules must be clear.

*San Jose mayor proposes gun owners carry insurance and pay annual fee in wake of mass shooting.*

I am in favor of this. Gun owners should also have to get safety training and keep the guns in ways that prevent theft or unapproved use of the gun.

Payments to COVAX vaccine system [Richard Stallman's Political Notes]

*Venezuela says payments to COVAX vaccine system have been blocked* by US sanctions.

Rep. Cori Bush demands records from FBI [Richard Stallman's Political Notes]

Rep. Cori Bush put FBI director Christopher Wray on the spot while he was testifying to Congress. She demanded a copy of the records that the FBI had collected on her protest activity.

More power to WTO [Richard Stallman's Political Notes]

Australia's government wants to give the WTO more power to make countries bow down to foreign businesses.

This is indeed the purpose of business-supremacy treaties, but few governments dare to admit they advocate that purpose.

Apparent favoritism [Richard Stallman's Political Notes]

Now that a court found "apparent favoritism" by ministers in choice of companies to get UK contracts, how will that make future decisions less corrupt?

The old UK method was that ministers had to resign when visibly tainted. Now, like US Republicans, Tories don't object to corrupt ministers, as long as they are Tories. So the question is whether this will weaken ministers' hand in disagreements with civil service professionals.

I don't have confidence in that.

Voluntourism [Richard Stallman's Political Notes]

*Voluntourism: new book explores how volunteer trips harm rather than help.*

Oregon house expels Republican [Richard Stallman's Political Notes]

*Oregon house expels Republican who helped far-right rioters enter capitol.*

That is the treatment Republicans bent on insurrection deserve.

Insufficient Covid vaccine donations [Richard Stallman's Political Notes]

The G7's pledges to donate an insufficient quantity of Covid vaccines constitute a total failure.

Unidentified aerial phenomena [Richard Stallman's Political Notes]

Unidentified aerial phenomena may appear to be "out of this world", but they are in fact mundane objects that appear mysterious due to the circumstances of the view.

None of them are evidence for alien visitors.

05:42

Still with the Kickstarter Sketches [Skin Horse]

Shaenon: More sketches for Kickstarter backers! Aimee’s outfit was inspired by Japanese mori girl fashion, which I low-key got into over quarantine and now I put decorative sticks in my hair.

Channing: I, too, frequently have sticks in my hair, but that’s just because there’s a lot of pine trees with dead lower branches around my house. Anyway, loving Big Mell and Sergio so much.

Saturday, 12 June

22:35

The airline mile hoax [Seth's Blog]

First: If you’re a frequent flyer on American and haven’t flown in over a year, it’s possible your miles are going to expire very soon. You can fix this by “donating” 2,500 miles here.

In the US, private lotteries are against the law. A lottery is a random drawing for a prize of value that you have to pay to participate in.

That’s different from a game of skill, in which the best performance wins.

Or a sweepstakes, which doesn’t cost anything to enter (which is why the rules so often say ‘no purchase necessary.’)

The question is: Is it a random/lucky thing to be able to trade in your miles for the prize you were promised? I think it’s pretty clear that as the points economy has gotten into the billions, the answer is yes. There aren’t as many ‘free’ seats per miles as there used to be. The airlines benefit when they offer fewer and fewer seats as a percentage of available points floating around, because then people are pushed to either ignore their miles or settle for something less than they expected.

Some people play with points as a hobby. For the rest of us, they’re worth way less than they appear. But mostly I wanted to remind you not to let yours expire. Thanks for staying safe by staying home.

UPDATE: After I queued up this post, AA and UA extended their deadlines. I’m glad! The rest of my rant still persists.

18:14

Fighting autocracy with marketing [Scripting News]

Joe Trippi's latest podcast focused me on realistic next steps fighting Republican autocracy. Here are my comments in a bulleted list. I wrote this quickly, so think of it as a memo, not a manifesto. ;-)

  • Joe, glad you're hooked up with the Lincoln Project people. They were doing well marketing against insurrection until the 2020 election.
  • The task they set themselves up to do is done. They were trying to convince Repubs it was ok to vote for a Democrat. That was great for the 2020 election but it's a small part of the campaign that should be running now.
  • They (or someone) must be ready to position the next huge insurrection event, with online advertising, in real time as it's happening, and in the immediate aftermath.
  • If such an operation were in existence on Jan 6, we wouldn't have had to wait for the DOJ to tell us what happened (they never have). Now, no amount of marketing will change that the events of Jan 6 are fuzzy and uncertain in our minds, even though in fact they aren't. No one focused our minds on the criminality of the president. We've never gotten an account of what the president was doing during the insurrection, yet the new administration must have that information? Why are we not running ads demanding that the information be released.
  • So we remain confused, no one has taken charge of communicating about that event. That's how the human mind works. It's why when Barr lied about the Mueller Report his spin stuck, even though later we knew he was lying.
  • The question the new Lincoln Project should always be raising: "Why is this traitor still in Congress? Repeat that theme over and over. Point the finger at the traitors, make them pay a political price for their insurrection.
  • When one of them does something really insidious, demand that they be removed from Congress right now. Put the question into the reports in NYT, WP, CNN, MSNBC, et al.
  • Jan 6 was not the last act of insurrection. Any day now there will be another. We must be ready to communicate about it.
  • If such a marketing channel existed, ordinary people like me will be able to do something instead of sitting around worrying and thinking no one will hear me so why bother. Our contribution will be in dollars to run ads, and feet marching in protest.
  • Marketers are in a position to do a lot of good. I think Lincoln Project should change their name and add Dems and independents to the project. It can't just be former Republicans at this point.
  • From now on campaigns never shut down. Tattoo that on your forehead. It's a mistake we're still making and it must stop. The need to communicate about events is persistent.

17:21

Pluralistic: 12 Jun 2021 [Pluralistic: Daily links from Cory Doctorow]


Today's links

  • The ACCESS Act: The most significant interop legislation in US history.
  • This day in history: 2006, 2011, 2016, 2020
  • Colophon: Recent publications, upcoming/recent appearances, current writing projects, current reading



EFF's interoperability banner graphic, a kind of Rube Goldberg machine integrating pulleys, belts, megaphones, emoticons, lightbulbs, HTML tags, a Creative Commons icon, a radio tower, a padlock, etc. >

The ACCESS Act (permalink)

Five Big Tech antitrust bills were introduced in the House Judiciary Committee today; they're the most significant antitrust effort in more than half a century, and they cover a lot of ground.

https://cicilline.house.gov/press-release/house-lawmakers-release-anti-monopoly-agenda-stronger-online-economy-opportunity

There's a bill to ban "self-preferencing" (when a company-run marketplace pushes its inferior products over its rivals' superior ones); another to block anticompetitive acquisitions; a bill to block "walled gardens"; and a bill to fund the FTC to police all this stuff.

But I'm most excited about is the ACCESS Act, a bill to force interoperability on the biggest tech platforms, the kinds of services people use because they have to, because their friends or communities or customers (or media) are locked into them.

https://www.eff.org/deeplinks/2021/06/access-act-takes-step-towards-more-interoperable-future

Under the ACCESS Act, very large companies will be required to offer an API that allows users to take their data to a rival service, and let them continue to talk to the people they left behind when they quit Big Tech.

It's designed to put an end to the Roach Motel business model, where your data checks in, but never checks out.

Now, interop is a great remedy for tech monopolies, but it comes with risks: first, that new platforms could abuse the data users bring with them, and second, that the API itself could be abused to steal data.

The ACCESS Act requires good security practices for the API and the services that connect to it, and it has a "circuit-breaker" cause allowing big platforms to temporarily shut off the API if someone figures out how to exploit it (and penalties for pretextual use of this).

And it has good – but not perfect – language protecting user privacy. The new services that take advantage of these new data flows are bound by rules prohibiting them from exploiting it or making money from it – and the law provides for stonking fines for rulebreakers.

Sure, "a fine is a price," but this is a steep price: the larger of 15% of total global revenue (not profit) or 30% of US revenues.

To get a sense of how privacy and interop can help each other check out today's techno-legal analysis from EFF:

https://www.eff.org/deeplinks/2021/06/gdpr-privacy-and-monopoly

Despite that, there's an important omission from this bill: a private right of action.

Laws with private rights of action can be enforced by the public – that is, if a company hurts you, you can sue (or join a class action, or seek help from a public interest lawfirm).

Without that private right of action, you have to hope that someone at the FTC, or an attorney general, or some other federal law enforcement entity decides to stick up for your rights.

Without a private right of action, the ACCESS Act depends on well-funded, motivated federal agencies.

The problem, of course, is that the GOP doesn't like private rights of action; they see them as "anti-business," and an invitation to "nuisance suits."

That matters because there's actually a chance that Republicans will support these. After decades of cheerleading for monopolies, the GOP was horrified to discover that the whole digital world is controlled by massive companies with weird, capricious moderation policies.

Once those policies were turned on them – after years of use against Palestinians, trans people, sex workers, BLM activists, Water Protectors, etc – Republicans stopped asserting the absolute right of businesses to set their own policies.

Most of the GOP policy responses to this are, frankly, really stupid, from state laws prohibiting platforms from terminating politicians' accounts to "common carrier" (no deleting porn!) and "fairness doctrine" (every astronomer has to debate an astrologist).

Far better than trying to turn the massive, unweildy monopolists into good monopolists is to end monopolies, creating a federated online world where users can choose the speech norms they're comfortable with and connect to users on other services.

That's not just a better policy, it's a better Republican policy. If you think it's a free speech violation to force a bakery to make a gay wedding cake, how the hell can you think it's OK to force a platform to carry your speech?

Which is to say, there's potential bipartisan support for these bills, and to secure it, the bills' sponsors are willing to set aside a private right of action.

I happen to think it's not a good trade-off, and I plan to campaign for these bills to pass – after they're amended to add a private right of action.

Despite the compromises, these are very good bills, the biggest thing to happen to antitrust in generations.



This day in history (permalink)

#15yrsago Images from anti-DRM protest at the San Fran Apple Store https://www.flickr.com/photos/quinn/tags/drmprotest/

#10yrsago Reasons people were arrested at the Toronto G20 <a 20110610125236="" href="https://memex.craphound.com/2011/06/11/reasons-people-were-arrested-at-the-toronto-g20/>https://memex.craphound.com/2011/06/11/reasons-people-were-arrested-at-the-toronto-g20/</a>

#10yrsago Ontario publicly funded Catholic school bans rainbows, appropriates student donations for LGBT cause and gives them to Catholic charity <a href=" http:="" https:="" public="" rainbows_banned_at_mississauga_catholic_school-10262.aspx"="" toronto="" web="" web.archive.org="" www.xtra.ca="">https://web.archive.org/web/20110610125236/http://www.xtra.ca/public/Toronto/Rainbows_banned_at_Mississauga_Catholic_school-10262.aspx

#5yrsago Mounties used Stingrays to secretly surveil millions of Canadians for years https://www.vice.com/en/article/kb73an/the-rcmp-surveilled-thousands-of-innocent-canadians-for-a-decade

#5yrsago How to be less wrong about the First Amendment https://www.popehat.com/2016/06/11/hello-youve-been-referred-here-because-youre-wrong-about-the-first-amendment/

#1yrago MIT dumps Elsevier https://pluralistic.net/2020/06/12/digital-feudalism/#nerdfight

#1yrago NY repeals 50-a https://pluralistic.net/2020/06/12/digital-feudalism/#hiding-in-plain-sight

#1yrago HBO Max, a monopolist's parable https://pluralistic.net/2020/06/12/digital-feudalism/#hbo-max

#1yrago Unauthorized water https://pluralistic.net/2020/06/12/digital-feudalism/#filtergate



Colophon (permalink)

Today's top sources:

Currently writing:

  • Spill, a Little Brother short story about pipeline protests. Friday's progress: 264 words (5483 words total).
  • A Little Brother short story about remote invigilation. PLANNING

  • A nonfiction book about excessive buyer-power in the arts, co-written with Rebecca Giblin, "The Shakedown." FINAL EDITS

  • A post-GND utopian novel, "The Lost Cause." FINISHED

  • A cyberpunk noir thriller novel, "Red Team Blues." FINISHED

Currently reading: Analogia by George Dyson.

Latest podcast: How To Destroy Surveillance Capitalism (Part 06) https://craphound.com/nonficbooks/destroy/2021/05/10/how-to-destroy-surveillance-capitalism-part-06/
Upcoming appearances:

Recent appearances:

Latest book:

Upcoming books:

  • The Shakedown, with Rebecca Giblin, nonfiction/business/politics, Beacon Press 2022

This work licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.


How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/web/accounts/303320

Medium (no ads, paywalled):

https://doctorow.medium.com/

(Latest Medium column: "The Rent’s Too Damned High," about the long con of convincing Americans that they will grow prosperous through housing wealth, not labor rights https://doctorow.medium.com/the-rents-too-damned-high-520f958d5ec5)

Twitter (mass-scale, unrestricted, third-party surveillance and advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla

16:42

Blogging from an outliner [Scripting News]

If you want to blog from an outliner, I suggest the first step is to get the outline data to flow out of your editing tool in OPML, and then write your publishing software to build the blog from the OPML. That way you won't be locked into your editor or publishing software.

There are lots of approaches to both writing and publishing. You won't get the benefit if you don't have a strong basis for interop.

Silos have a long history of being bad for progress.

For most of the last 26 years I've been writing my blog in an outliner. I'm glad that other people are interested in this now. Let's build a new community around this idea. ;-)

The rest is in a Twitter thread.

14:42

Norbert Preining: Future of Cinnamon in Debian [Planet Debian]

OK, this is not an easy post. I have been maintaining Cinnamon in Debian for quite some time, since around the times version 4 came out. The soon (hahaha) to be released Bullseye will carry the last release of the 4-track, but version 5 is already waiting, After Bullseye, the future of Cinnamon in Debian currently looks bleak.

Since my switch to KDE/Plasma, I haven’t used Cinnamon in months. Only occasionally I tested new releases, but never gave them a real-world test. Having left Gnome3 for it’s complete lack of usability for pro-users, I escaped to Cinnamon and found a good home there for quite some time – using modern technology but keeping user interface changes conservative. For long time I haven’t even contemplated using KDE, having been burned during the bad days of KDE3/4 when bloat-as-bloat-can-be was the best description.

What revelation it was that KDE/Plasma was more lightweight, faster, responsive, integrated, customizable, all in all simple great. Since my switch to KDE/Plasma I think not for a second I have missed anything from the Gnome3 or Cinnamon world.

And that means, I will most probably NOT packaging Cinnamon 5, nor do any real packaging work of Cinnamon for Debian in the future. Of course, I will try to keep maintenance of the current set of packages for Bullseye, but for the next release, I think it is time that someone new steps in. Cinnamon packaging taught me a lot on how to deal with multiple related packages, which is of great use in the KDE packaging world.

If someone steps forward, I will surely be around for support and help, but as long as nobody takes the banner, it will mean the end of Cinnamon in Debian.

Please contact me if you are interested!

13:56

Kentaro Hayashi: fabre.debian.net has moved to Debian.net Team Infrastructure [Planet Debian]

Today, fabre.debian.net has moved to Debian.net Team Infrastructure

So far, fabre.debian.net was sponsored by FOSSHOST which provides us a VPS instance since Jan, 2021. It was located at OSU Open Source Lab. It worked pretty well, Thanks FOSSHOST sponsorship since ever!

Now, fabre.debian.net uses the VPS instance which is provided by Debian.net Team Infrastructure. (still non-DSA managed) It is hosted at HETZNER Cloud.

About fabre.debian.net

fabre.debian.net is a experimental service to demonstrate how to improve user experience with finding and fixing Debian unstable related bugs for making "unstable life" comfortable.

Thank Debian.net Team for sponsoring,

10:07

Disenchantment [Seth's Blog]

It originally means, “no longer believing in magic.”

Humans like magic. It gives us solace and energy and hope.

In many ways, the rational era of science and engineering and evidence and proof eliminated any practical belief in magical forces. We know how and why the sun sets every night.

But we still desire magic.

Creating it for your customers and peers is a gift.

10:00

Junichi Uekawa: Wrote a quick hack to open chroot in emacs tramp. [Planet Debian]

Wrote a quick hack to open chroot in emacs tramp. I wrote a mode for cros_sdk and it was relatively simple. I figured that chroot must be easier. I could write one in about 30 minutes. I need to mount proc and home inside the chroot to make it useful, but here goes. chroot-tramp.el

02:14

Google ends its attack on the URL bar, resumes showing full address in Chrome [OSnews]

As Android Police reports:

Google has tried multiple times for years to dumb down the internet by simplifying Chrome’s “scary” address bar. It first tried to erode the URL entirely by showing just search terms in the omnibox, but its impractical design forced Google to retire it. The developers recently tried to simplify the omibox again — this time hiding all parts of the web address except the domain name. While it received a fair amount of criticism from users, Google defended its decision to move forward, citing its intention to help people better identify malicious sites. But now it seems that Google has reconsidered things, as it recently decided to close the curtains on its experiment.

Good. URLs present important information, and preventing or limiting access to it is simply dumb, and asking for trouble.

Public key cryptography: OpenSSH private keys [OSnews]

When you create standard RSA keys with ssh-keygen you end up with a private key in PEM format, and a public key in OpenSSH format. Both have been described in detail in my post Public key cryptography: RSA keys. In 2014, OpenSSH introduced a custom format for private keys that is apparently similar to PEM but is internally completely different. This format is used by default when you create ed25519 keys and it is expected to be the default format for all keys in the future, so it is worth having a look.

An in-depth analysis of what’s inside the OpenSSH private key format and how it is different from the standard PEM format.

00:07

[$] Code humor and inclusiveness [LWN.net]

Free-software development is meant to be fun, at least some of the time. Even developers of database-management systems seem to think that it is fun; there is no accounting for taste, it seems. Part of having fun is certainly allowing the occasional exercise of one's sense of humor while working on the code. But, as some recent "fix" attempts show, humor does not always carry through to developers all over the planet. Balancing humor and inclusiveness is always going to be a challenge for our community.

Privacy analysis of FLoC (Mozilla blog) [LWN.net]

Over on the Mozilla blog, Eric Rescorla looks into some of the privacy implications of the Federated Learning of Cohorts (FLoC), which is a Google effort to replace third-party cookies with a different type of identifier that is less trackable. But less tracking does not equal no tracking. "People's interests aren't constant and neither are their FLoC IDs. Currently, FLoC IDs seem to be recomputed every week or so. This means that if a tracker is able to use other information to link up user visits over time, they can use the combination of FLoC IDs in week 1, week 2, etc. to distinguish individual users. This is a particular concern because it works even with modern anti-tracking mechanisms such as Firefox's Total Cookie Protection (TCP). TCP is intended to prevent trackers from correlating visits across sites but not multiple visits to one site. FLoC restores cross-site tracking even if users have TCP enabled."

Friday, 11 June

23:21

Poettering: The Wondrous World of Discoverable GPT Disk Images [LWN.net]

In a lengthy blog post, Lennart Poettering describes the advantages of using the unique IDs (UUIDs) and flags from the discoverable partitions specification to label the entries in a GUID Partition Table (GPT). That information can be used to tag disk images in a self-descriptive way, so that external configuration files (such as /etc/fstab) are not needed to assemble the filesystems for the running system. Systemd can use this information in a variety of ways, including for running the image in a container: "If a disk image follows the Discoverable Partition Specification then systemd-nspawn has all it needs to just boot it up. Specifically, if you have a GPT disk image in a file foobar.raw and you want to boot it up in a container, just run systemd-nspawn -i foobar.raw -b, and that's it (you can specify a block device like /dev/sdb too if you like). It becomes easy and natural to prepare disk images that can be booted either on a physical machine, inside a virtual machine manager or inside such a container manager: the necessary meta-information is included in the image, easily accessible before actually looking into its file systems."

23:07

Mike Gabriel: New: The Debian BBB Packaging Team (and: Kurento Media Server goes Debian) [Planet Debian]

Today, Fre(i)e Software GmbH has been contracted for packaging Kurento Media Server for Debian. This packaging project will be funded by GUUG e.V. (the German Unix User Group e.V.). A big thanks to the people from GUUG e.V. for making this packaging project possible.

About Kurento Media Server

Kurento is an open source software project providing a platform suitable for creating modular applications with advanced real-time communication capabilities. For knowing more about Kurento, please visit the Kurento project website: https://www.kurento.org.

Kurento is part of FIWARE. For further information on the relationship of FIWARE and Kurento check the Kurento FIWARE Catalog Entry. Kurento is also part of the NUBOMEDIA research initiative.

Kurento Media Server is a WebRTC-compatible server that processes audio and video streams, doing composable pipeline-based processing of media.

About BigBlueButton

As some of you may know, Kurento Media Server is one of the core components of the BigBlueButton software, an ,,Open Source Virtual Classroom Software''.

The context of the KMS funding is - after several other steps - getting the complete software component stack of BigBlueButton (aka BBB) into Debian some day, so that we can provide BBB as native Debian packages. On Debian. (Currently, one needs to use an always already a bit outdated version of Ubuntu).

Due to this greater context, I just created the Debian BBB Packaging Team on salsa.debian.org.

Outlook and Appreciation

The current project (uploading Kurento Media Server to Debian) will very likely be extended to one year of package maintenance for all Kurento Media Server components in Debian. Extending this maintenance funding to a second year, has also been discussed, and seems a possible option.

Probably most Debian Developer colleagues will agree with me when I say that Debian packaging is not a one-time shot until the first uploads of software packages have landed and settled. Debian package maintenance is a long term responsibility and requires long term commitment. I am very glad, that the people at GUUG e.V are on the same page with me (with us) regarding this. This is much and dearly appreciated. Thank you!!!

What else?

Well, we have also talked about another BigBlueButton component that is not yet in Debian: FreeSwitch. But more of that, when time has come.

How to Join the Debian BBB Packaging Team?

Please ping me via IRC (sunweaver on OFTC IRC) or [matrix] (@sunweaver:matrix.org).

How to Support the Debian BBB Packaging Team?

If you, your organization, your company, your municipality, your university, etc. feels like supporting the effort of packaging BigBlueButton for Debian, please get in touch with: mike.gabriel@freiesoftware.gmbh

And yes, the company homepage is not online, yet, but it is in the makings...

light+love
Mike (aka sunweaver)

22:35

Friday Squid Blogging: Fossil of Squid Eating and Being Eaten [Schneier on Security]

We now have a fossil of a squid eating a crustacean while it is being eaten by a shark.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Read my blog posting guidelines here.

22:28

News Post: He-Mania [Penny Arcade]

Tycho: I genuinely don't know a lot about He-Man, although my friend Damon had a Battle Armor He-Man whose inner workings perpetually mystified me. You could strike the breastplate, and every time it would reveal a progressively more damaged version until it was perfect again. What a terrifying foe. Strike your enemy dead in the chest three times to… perfect him? I'm not fucking fighting this guy. That explains the raft of robots, weird snake guys, and bipedal insects he's has to contend with. He's almost certainly murdered everyone else, except for the one guy whose face…

22:21

Page 38 [Flipside]

Page 38 is done.

21:35

21:00

Link [Scripting News]

Today's song: Oklahoma USA.

Link [Scripting News]

Joe Trippi did a podcast in response to my podcast. This time I listened all the way through. I like that Joe is open to other points of view. This is an important time for that. I'm also glad he's talking with the Lincoln Project. I'm going to think about this for a bit and then respond.

20:49

“Getting Back To Normalcy” [Whatever]

Athena ScalziFucking Christ on a cheese stick, I am so tired of that phrase.

If I had a dime for every time I’ve heard, “as the world opens back up”, “as we return to normality”, or “as things are getting back to normal,” I wouldn’t have to live with my parents.

Companies love something they can “relate” to their customers about. Companies love heartfelt concepts and wholesome ideas they can use to market to their demographics. And they love pretending like they care.

And what gives them a better excuse to pretend like they care than the biggest pandemic of our lifetime?

How many car commercials did you see during the pandemic that started with, “in times of uncertainty”? How many cereal commercials said, “we’ll get through this together?”

After writing that sentence, I Googled “commercials that said ‘in times of uncertainty” and it turns out there’s like actual articles about this phenomenon! Here’s the Wall Street Journal saying that these commercials have a “tragedy template”. This article is from one month into the pandemic. Over a year later, companies are still being as annoying as all hell, but now they’ve shifted from “we’re in this together during this uncertain time” to “as you start to go back outside and are now visiting businesses and spending money again.”

This is literally what they sound like:

Also posted over a year ago. But they just keep coming! Companies keep busting out these “heartfelt” and “compassionate” commercials even though nobody asked for them in the first place.

I don’t want companies to act like they care. It’s just embarrassing on their part. Everyone knows they’re only in it for the money. You know how it’s evident? Because they’re still trying to sell you shit during the pandemic. It doesn’t matter how they frame it, even if they say that times are hard and that they care, they still want you to give them money. If they really cared, would they even advertise?

This idea of “returning to normal” is even more problematic than the insincere, copy and paste, “sad” commercials that companies were doing for months.

This whole “returning to normal” thing isn’t just company and commercial related, though. It’s workplace and school-related, too. The “returning to normal” ideology is toxic for institutions to have, because we aren’t just “going back to normal”. We can’t just shrug it off and go back to how things were.

The problem with these institutions is that they think we’ll just get over it. The pandemic is over now, right? People are getting vaccinated, we don’t have to wear masks anymore, it’s all hunky dory, right? But what these institutions don’t understand is trauma. They can’t see the long-term effects.

The pandemic has changed everything, yet we are expected to return to how things were before. But how can we? These institutions, as well as companies, cannot understand how profoundly the pandemic has affected not just society, but people on the individual scale.

Part of that is because they don’t want things to change. Like I said, the pandemic has changed everything, but what do I mean by that? Because, from the looks of it, almost nothing has actually changed. For example, aside from the vaccine, did we get tax-supported no-cost healthcare? That would have helped. Did we get the institutional level of support that would have been equal to these “uncertain times”? No.

And why would these institutions allow any sort of change when things have been working so well for them up to this point? They’re not going to suddenly turn around and be like, “oh, we’ve realized our mistakes and now see the flaws in the systems we’ve created” because they’ve known all along. They know their systems are fucked up, but it makes them money so why would they stop?

Meanwhile, we as individuals, are completely changed. Maybe you’ve lost loved ones, or lost your job, maybe even lost your home, or got COVID and suffered serious effects. Or maybe nothing really happened to you personally, but you got a front row seat to watch the world around you burn, and that’s traumatizing enough on its own.

Most of us have known for a while that our society and our government are fucked up, but this pandemic really put the final nail in the coffin. It was eye-opening for a lot of people. It showed that we are not cared about, even if a Ford commercial says we are. It showed that our institutions would rather sacrifice us than lose money. And it showed that we would rather sacrifice each other than not go out to eat at Applebee’s.

Yes, things are returning to normal. But only after half a million people died, only after the unemployment rate skyrocketed to new heights, only after the homelessness rate increased, and only after we’ve all sustained trauma that we will carry with us for the rest of our lives.

But we aren’t feeling, thinking, human beings to these institutions. We are numbers on a graph, we are statistics in the making, we are cogs in the machine. They couldn’t care less if our mental health is less than fucking ideal after over a year of dealing with the pandemic. We are meant only to make them profit, traumatized or not. Depressed or not. Anxious or not. Burnt out or not.

So, I’d really appreciate if companies stopped acting like we’re in this together. Because we aren’t, and we never were.

-AMS

20:00

Lisandro Damián Nicanor Pérez Meyer: Firsts steps into QML [Planet Debian]

After years of using and maintaining Qt there was a piece of the SDK that I never got to use as a developer: QML. Thanks to ICS I've took the free (in the sense of cost) QML Programming — Fundamentals and Beyond.

It consists of seven sessions, which can be easily done in a few days. I did them all in 4 days, but with enough time available you can do them even faster. Of course some previous knowledge of Qt comes handy.

The only drawback was the need of a corporate e-mail in order to register (or at least the webpage says so). Apart from that it is really worth the effort. So, if you are planning into getting into QML this is definitely a nice way to start.

19:07

Unleash your inner explorer! [Humble Bundle Blog]

 It’s time to get out there and explore! Fortunately, you don’t need to leave the comfort of your gaming rig

Continue reading

The post Unleash your inner explorer! appeared first on Humble Bundle Blog.

18:07

Pluralistic: 11 Jun 2021 [Pluralistic: Daily links from Cory Doctorow]


Today's links



EFF's interoperability banner graphic, a kind of Rube Goldberg machine integrating pulleys, belts, megaphones, emoticons, lightbulbs, HTML tags, a Creative Commons icon, a radio tower, a padlock, etc.

Privacy Without Monopoly, EU edition (permalink)

Tech monopoly apologists insist that there's something exceptional about tech that makes it so concentrated: "network effects" (when a product gets better because more people use it, like a social media service).

They're wrong.

Tech is concentrated because the Big Tech companies buy up or crush their nascent competitors – think of Facebook's predatory acquisition of Instagram, which Zuckerberg admitted (in writing!) was driven by a desire to recapture the users who were leaving FB in droves.

Google's scale is driven by acquisitions – Search and Gmail are Google's only successful in-house products. Everything else, from Android to Youtube to their entire ad-tech stack, was once a standalone business that Google captured.

Monopolies extract monopoly rents – like those delivered by Googbook's crooked ad-tech marketplaces, or Apple/Google's 30% app shakedown – and use them to maintain their monopolies. Google gives Apple billions every year so it will be the default Ios and Safari search.

These are the same tactics that every monopolist uses – high-stakes moneyball that creates a "kill-zone" around the monopolist's line of business that only a fool would try to enter. Tech DOES have network effects, but that's not what's behind tech monopolies.

We see monopolies in industries from bookselling to eyeglasses, accounting to cheerleading uniforms, pro wrestling to energy, beer to health insurance. These monopolies all follow Big Tech's template of mobilizing monopoly rents to buy or crush all competition.

The differences between the anticompetitive tactics that monopolized these industries are largely cosmetic – swap out a few details and you might well be describing how John D Rockefeller and Standard Oil monopolized the oil markets in the late 19th and early 20th centuries.

Big Tech does have network effects, but these are actually a tool that can be used to dismantle monopolies, as well as maintaining them. Network effects are double-edged swords: if a service gets more valuable as users join, it also gets less valuable as users leave.

If you want to understand the anticompetitive structure of the tech industry, you'd be better off analyzing switching costs, not network effects. Switching costs are the things you have to give up when you leave a service behind.

If your customers, community, family members or annotated photos and other memories are locked up in Facebook's walled garden (or if you've got money sunk in proprietary media or apps on Apple's, etc), then the switching cost is losing access to all of that.

Here's where tech really is different: tech has intrinsically low switching costs. Latent in all digital technology is the capacity to interoperate, to plug a new service into an old one, to run an old app inside a simulator ("runtime").

There's no good technical reason you can't leave Facebook but take your treasured photos with you – and continue to exchange messages with the people you left behind.

True, Facebook has gone to extraordinary lengths to keep its switching costs high, deploying technical countermeasures to block interoperability. But these aren't particularly effective. Lots of people have figured out how to reverse-engineer FB and plug new things into it.

Power Ventures created an app that aggregated your FB feed with feeds from rival services, giving you a single dashboard. NYU's Ad Observer scrapes the political ads FB shows you for analysis to check whether FB is enforcing its own paid political disinformation rules.

And there's a whole constellation of third-party Whatsapp clients that add features FB has decided Whatsapp users don't deserve, like the ability to block read-receipts or run multiple accounts on the same device.

https://www.eff.org/deeplinks/2020/03/african-whatsapp-modders-are-masters-worldwide-adversarial-interoperability

Most of these are technical successes, but they're often legal failures. FB has used the monopoly rents it extracted to secure radical new laws and new interpretations of existing laws to make these tactics illegal.

Power Ventures was sued into oblivion. Ad Observer is fighting for its life. The Whatsapp mods are still going strong, but that may be down to the jurisdictions where they thrive – sub-Saharan Africa – where FB has less legal muscle.

With low switching costs, much of FB's monopoly protection evaporates. Lots of people hate FB, and FB knows it. You're on FB because your friends are there. Your friends are there because you're there. You've taken each other hostage, and FB benefits.

With low switching costs, you could leave FB – but not your friends. The kill zone disappears. All we need is interoperability.

Enter the EU's Digital Services Act and Digital Markets Act, proposed regulations to force interop on the biggest Big Tech players.

The EU has recognized that mandating interop can reduce switching costs, and reducing switching costs can weaken monopoly power.

Some critics (like me!) of the EU proposals say they don't go far enough, asking for "full interop" for rival services.

Against these calls for broader interop come warnings about the privacy implications of forcing FB to open up its servers to rivals. It's hard enough to keep FB from abusing its users' privacy, how will we keep track of a constellation of services that can access user data?

Last Feb, Bennett Cyphers and I published "Privacy Without Monopoly," for EFF, describing how interoperability can enhance privacy.

Interop means that users can choose services that have better privacy policies than Facebook or other incumbent platforms.

https://www.eff.org/wp/interoperability-and-privacy

But in theory, it means that users could choose worse services – services that have worse privacy policies, services that might be able to grab your friends' data along with your own (say, the pictures you took of them and brought with you, or their private messages to you).

That's why, in our paper, we say that interop mandates have to be backstopped by privacy rules – democratically accountable rules from lawmakers or regulators, not self-serving "privacy" limitations set by the Big Tech companies themselves.

For example, Facebook aggressively imports your address books when you sign up, to connect you to the people you know (this isn't always a good experience – say, if your stalker has you in their address book and automatically gets "friended" with you).

If you try to take your address book with you when you quit, FB claims your contact list isn't "yours" – it belongs to your contacts. To protect their privacy, FB has to block you from exporting the data – making it it much harder to establish social ties on a new service.

It's not obvious who that contact info "belongs to" (if "belong to" is even the right way to talk about private information that implicates multiple people!).

But what is obvious is that Facebook can't be trusted to make that call.

Not only has Facebook repeatedly disqualified itself from being trusted to defend its users' privacy, but it also has a hopeless conflict of interest, because privacy claims can be used to raise switching costs and shore up its monopoly.

In our paper, Bennett and I say that these thorny questions should be resolved democratically, not in a corporate boardroom.

Now, as it happens, there's a region where 500M people are protected by a broad, democratically enacted privacy law: Europe, home of the GDPR.

Today, in a new appendix to "Privacy Without Monopoly," EFF has published "The GDPR, Privacy and Monopoly," my analysis of how the GDPR makes interoperability safer from a privacy perspective.

https://www.eff.org/deeplinks/2021/06/gdpr-privacy-and-monopoly

Working with EFF's Christoph Schmon and Bennett Cyphers, and Ala Krinickytė, a data protection lawyer at NOYB, we develop a detailed analysis of the GDPR, and describe how the GDPR provides a lawful framework for resolving thorny questions about consent and blended title to data.

The GDPR itself seeks to promote interoperability; it's right there in Recital 68: "data controllers should be encouraged to develop interoperable formats that enable data portability." But loopholes in the rules have allowed dominant companies to stymie interop.

For years, Europeans have had the "right" to port their data, but nowhere to port that data to. The DMA closes the loopholes and dismantles the hurdles that kept switching costs high.

The GDPR's consent/security/minimization framework sets out the parameters for any interoperability, meaning we don't have to trust Facebook (or Google, or Amazon, or Apple) to decide when interop must be blocked "to defend users' privacy" (and also shareholders' profits).

Big Tech platforms already have consent mechanisms (and must continue to build them) to create the legal basis for processing user data. An interoperable FB could be a consent conduit, letting your friends decide when and whether you can take their data to a new service.

And the GDPR (not a tech executive) also determines when a new service meets the privacy standards needed for interop. It governs how that new service must handle user data, and it gives users a way to punish companies that break the rules.

Today, if you leave Facebook, your friends might not even notice. But in a world where FB is a consent conduit to manage your departure and resettlement, all your friends get signals about your departure – perhaps prompting them to consider whether they should go, too.

Far from prohibiting interop, the GDPR enables it, by creating an explicit privacy framework that is consistent across all services, both the old monopolies and the new co-ops, startups, public utilities, and other alternatives that interop would make possible.

Monopolies distort the world in two ways. The most obvious harm is to competition, choking out or buying out every alternative, so you have to live by whatever rules the monopolist sets.

But the other kind of harm is even worse: monopolists can use their political power to get away with terrible abuses.

Ad-tech concentration produced monopoly rents that blocked or weakened privacy law for decades, allowing for a grotesque degree of commercial surveillance.

We don't want competition in surveillance.

Opening space for interop poses a legitimate risk of creating a contest to see who can violate your human rights most efficiently.

https://pluralistic.net/2021/06/08/leona-helmsley-was-a-pioneer/#monkeys-paw

Yet, it's obvious that monopolists themselves shouldn't get to decide where they should be subjected to competition and where they should be subjected to regulation. That's a job for democratic institutions, not autocratic board-rooms.

Adding privacy regulation (strong privacy regulation, with a private right of action allowing users to sue companies for breaking the rules) to interop is how we resolve this conundrum, how we make sure we're banning surveillance, rather than "democratizing" it.



This day in history (permalink)

#20yrsago Embarassing gaffe in Microsoft’s anti-Linux campaign https://web.archive.org/web/20010619154332/http://www.newsforge.com/article.pl?sid=01/06/01/1540231

#15yrsago Inside China’s iPod sweat-shops https://www.macworld.co.uk/news/inside-apples-ipod-factories-14915/

#10yrsago Walt Disney’s 1956 time-capsule letter to the future https://web.archive.org/web/20060906113727/http://craphound.com.nyud.net:8080/walts1956letter.pdf

#10yrsago Terry Pratchett initiates assisted suicide process https://web.archive.org/web/20110614215922/http://www.telegraph.co.uk/health/8571142/Sir-Terry-Pratchett-begins-process-that-could-lead-to-assisted-suicide.htmln

#5yrsago It’s getting harder and harder to use gag clauses to silence laid off workers in America https://www.nytimes.com/2016/06/12/us/laid-off-americans-required-to-zip-lips-on-way-out-grow-bolder.html

#1yrago Interoperability and privacy https://pluralistic.net/2020/06/11/1-in-3/#interoperable

#1yrago Tesla modder selling discounted upgrades https://pluralistic.net/2020/06/11/1-in-3/#boost-50



Colophon (permalink)

Today's top sources:

Currently writing:

  • Spill, a Little Brother short story about pipeline protests. Yesterday's progress: 253 words (5218 words total).
  • A Little Brother short story about remote invigilation. PLANNING

  • A nonfiction book about excessive buyer-power in the arts, co-written with Rebecca Giblin, "The Shakedown." FINAL EDITS

  • A post-GND utopian novel, "The Lost Cause." FINISHED

  • A cyberpunk noir thriller novel, "Red Team Blues." FINISHED

Currently reading: Analogia by George Dyson.

Latest podcast: How To Destroy Surveillance Capitalism (Part 06) https://craphound.com/nonficbooks/destroy/2021/05/10/how-to-destroy-surveillance-capitalism-part-06/
Upcoming appearances:

Recent appearances:

Latest book:

Upcoming books:

  • The Shakedown, with Rebecca Giblin, nonfiction/business/politics, Beacon Press 2022

This work licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.


How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/web/accounts/303320

Medium (no ads, paywalled):

https://doctorow.medium.com/

(Latest Medium column: "The Rent’s Too Damned High," about the long con of convincing Americans that they will grow prosperous through housing wealth, not labor rights https://doctorow.medium.com/the-rents-too-damned-high-520f958d5ec5)

Twitter (mass-scale, unrestricted, third-party surveillance and advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla

17:21

Reproducible data processing pipelines [Planet GNU]

Last week, we at Guix-HPC published videos of a workshop on reproducible software environments we organized on-line. The videos are well worth watching—especially if you’re into reproducible research, and especially if you speak French or want to practice. This post, though, is more of a meta-post: it’s about how we processed these videos. “A workshop on reproducibility ought to have a reproducible video pipeline”, we thought. So this is what we did!

From BigBlueButton to WebM

Over the last year and half, perhaps you had the “opportunity” to participate in an on-line conference, or even to organize one. If so, chances are that you already know BigBlueButton (BBB), the free software video conferencing suite initially designed for on-line teaching. In a nutshell, it allows participants to chat (audio, video, and keyboard), and speakers can share their screen or a PDF slide deck. Organizers can also record the session.

BBB then creates a link to recorded sessions with a custom JavaScript player that replays everything: typed chat, audio and video (webcams), shared screens, and slide decks. This BBB replay a bit too rough though and often not the thing you’d like to publish after the conference. Instead, you’d rather do a bit of editing: adjusting the start and end time of each talk, removing live chat from what’s displayed (which allows you to remove info that personally identifies participants, too!), and so forth. Turns out this kind of post-processing is a bit of work, primarily because BBB does “the right thing” of recording each stream separately, in the most appropriate form: webcam and screen shares are recorded as separate videos, chat is recorded as text with timings, slide decks is recorded as a bunch of PNGs plus timings, and then there’s a bunch of XML files with metadata putting it all together.

Anyway, with a bit of searching, we quickly found the handy bbb-render tool, which can first download all these files and then assemble them using the Python interface to the GStreamer Editing Services (GES). Good thing: we don’t have to figure out all these things; we “just” have to run these two scripts in an environment with the right dependencies. And guess what: we know of a great tool to control execution environments!

A “deployment-aware Makefile”

So we have a process that takes input files—those PNGs, videos, and XML files—and produces output files—WebM video files. As developers we immediately recognize a pattern and the timeless tool to deal with it: make. The web already seems to contain countless BBB post-processing makefiles (and shell scripts, too). We were going to contribute to this while we suddenly realized that we know of another great tool to express such processes: Guix! Bonus: while a makefile would address just the tip of the iceberg—running bbb-render—Guix can also take care of the tedious task of deploying the right environment to run bbb-render in.

What we did was to write some sort of a deployment-aware makefile. It’s still a relatively unconventional way to use Guix, but one that’s very convenient. We’re talking about videos, but really, you could use the same approach for any kind of processing graph where you’d be tempted to just use make.

The end result here is a Guix file that returns a manifest—a list of videos to “build”. You can build the videos with:

guix build -m render-videos.scm

Overall, the file defines a bunch of functions (procedures in traditional Scheme parlance), each of which takes input files and produces output files. More accurately, these functions returns objects that describe how to build their output from the input files—similar to how a makefile rule describes how to build its target(s) from its prerequisite(s). (The reader familiar with functional programming may recognize a monad here, and indeed, those build descriptions can be thought of as monadic values in a hypothetical “Guix build” monad; technically though, they’re regular Scheme values.)

Let’s take a guided tour of this 300-line file.

Rendering

The first step in this file describes where bbb-render can be found and how to run it to produce a GES “project” file, which we’ll use later to render the video:

(define bbb-render
  (origin
    (method git-fetch)
    (uri (git-reference (url "https://github.com/plugorgau/bbb-render")
                        (commit "a3c10518aedc1bd9e2b71a4af54903adf1d972e5")))
    (file-name "bbb-render-checkout")
    (sha256
     (base32 "1sf99xp334aa0qgp99byvh8k39kc88al8l2wy77zx7fyvknxjy98"))))

(define rendering-profile
  (profile
   (content (specifications->manifest
             '("gstreamer" "gst-editing-services" "gobject-introspection"
               "gst-plugins-base" "gst-plugins-good"
               "python-wrapper" "python-pygobject" "python-intervaltree")))))

(define* (video-ges-project bbb-data start end
                            #:key (webcam-size 25))
  "Return a GStreamer Editing Services (GES) project for the video,
starting at START seconds and ending at END seconds.  BBB-DATA is the raw
BigBlueButton directory as fetched by bbb-render's 'download.py' script.
WEBCAM-SIZE is the percentage of the screen occupied by the webcam."
  (computed-file "video.ges"
                 (with-extensions (list (specification->package "guile-gcrypt"))
                  (with-imported-modules (source-module-closure
                                          '((guix build utils)
                                            (guix profiles)))
                    #~(begin
                        (use-modules (guix build utils) (guix profiles)
                                     (guix search-paths) (ice-9 match))

                        (define search-paths
                          (profile-search-paths #+rendering-profile))

                        (for-each (match-lambda
                                    ((spec . value)
                                     (setenv
                                      (search-path-specification-variable
                                       spec)
                                      value)))
                                  search-paths)

                        (invoke "python"
                                #+(file-append bbb-render "/make-xges.py")
                                #+bbb-data #$output
                                "--start" #$(number->string start)
                                "--end" #$(number->string end)
                                "--webcam-size"
                                #$(number->string webcam-size)))))))

First it defines the source code location of bbb-render as an “origin”. Second, it defines rendering-profile as a “profile” containing all the packages needed to run bbb-render’s make-xges.py script. The specification->manifest procedure creates a manifest from a set of packages specs, and likewise specification->package returns the package that matches a given spec. You can try these things at the guix repl prompt:

$ guix repl
GNU Guile 3.0.7
Copyright (C) 1995-2021 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guix-user)> ,use(guix profiles)
scheme@(guix-user)> ,use(gnu)
scheme@(guix-user)> (specification->package "guile@2.0")
$1 = #<package guile@2.0.14 gnu/packages/guile.scm:139 7f416be776e0>
scheme@(guix-user)> (specifications->manifest '("guile" "gstreamer" "python"))
$2 = #<<manifest> entries: (#<<manifest-entry> name: "guile" version: "3.0.7" …> #<<manifest-entry> name: "gstreamer" version: "1.18.2" …> …)

Last, it defines video-ges-project as a function that takes the BBB raw data, a start and end time, and produces a video.ges file. There are three key elements here:

  1. computed-file is a function to produce a file, video.ges in this case, by running the code you give it as its second argument—the recipe, in makefile terms.
  2. The recipe passed to computed-file is a G-expression (or “gexp”), introduced by this fancy #~ (hash tilde) notation. G-expressions are a way to stage code, to mark it for eventual execution. Indeed, that code will only be executed if and when we run guix build (without --dry-run), and only if the result is not already in the store.
  3. The gexp refers to rendering-profile, to bbb-render, to bbb-data and so on by escaping with the #+ or #$ syntax (they’re equivalent, unless doing cross-compilation). During build, these reference items in the store, such as /gnu/store/…-bbb-render, which is itself the result of “building” the origin we’ve seen above. The #$output reference corresponds to the build result of this computed-file, the complete file name of video.ges under /gnu/store.

That’s quite a lot already! Of course, this real-world example is more intimidating than the toy examples you’d find in the manual, but really, pretty much everything’s there. Let’s see in more detail at what’s inside this gexp.

The gexp first imports a bunch of helper modules with build utilities and tools to manipulate profiles and search path environment variables. The for-each call iterates over search path environment variables—PATH, PYTHONPATH, and so on—, setting them so that the python command is found and so that the needed Python modules are found.

The with-imported-modules form above indicates that the (guix build utils) and (guix profiles) modules, which are part of Guix, along with their dependencies (their closure), need to be imported in the build environment. What about with-extensions? Those (guix …) module indirectly depend on additional modules, provided by the guile-gcrypt package, hence this spec.

Next comes the ges->webm function which, as the name implies, takes a .ges file and produces a WebM video file by invoking ges-launch-1.0. The end result is a video containing the recording’s audio, the webcam and screen share (or slide deck), but not the chat.

Opening and closing

We have a WebM video, so we’re pretty much done, right? But… we’d also like to have an opening, showing the talk title and the speaker’s name, as well as a closing. How do we get that done?

Perhaps a bit of a sledgehammer, but it turns out that we chose to produce those still images with LaTeX/Beamer, from these templates.

We need again several processing steps:

  1. We first define the latex->pdf function that takes a template .tex file, a speaker name and title. It copies the template, replaces placeholders with the speaker name and title, and runs pdflatex to produce the PDF.
  2. The pdf->bitmap function takes a PDF and returns a suitably-sized JPEG.
  3. image->webm takes that JPEG and invokes ffmpeg to render it as WebM, with the right resolution, frame rate, and audio track.

With that in place, we define a sweet and small function that produces the opening WebM file for a given talk:

(define (opening title speaker)
  (image->webm
   (pdf->bitmap (latex->pdf (local-file "opening.tex") "opening.pdf"
                            #:title title #:speaker speaker)
                "opening.jpg")
   "opening.webm" #:duration 5))

We need one last function, video-with-opening/closing, that given a talk, an opening, and a closing, concatenates them by invoking ffmpeg.

Putting it all together

Now we have all the building blocks!

We use local-file to refer to the raw BBB data, taken from disk:

(define raw-bbb-data/monday
  ;; The raw BigBlueButton data as returned by './download.py URL', where
  ;; 'download.py' is part of bbb-render.
  (local-file "bbb-video-data.monday" "bbb-video-data"
              #:recursive? #t))

(define raw-bbb-data/tuesday
  (local-file "bbb-video-data.tuesday" "bbb-video-data"
              #:recursive? #t))

No, the raw data is not in the Git repository (it’s too big and contains personally-identifying information about participants), so this assumes that there’s a bbb-video-data.monday and a bbb-video-data.tuesday in the same directory as render-videos.scm.

For good measure, we define a <talk> data type:

(define-record-type <talk>
  (talk title speaker start end cam-size data)
  talk?
  (title     talk-title)
  (speaker   talk-speaker)
  (start     talk-start)           ;start time in seconds
  (end       talk-end)             ;end time
  (cam-size  talk-webcam-size)     ;percentage used for the webcam
  (data      talk-bbb-data))       ;BigBlueButton data

… such that we can easily define talks, along with talk->video, which takes a talk and return a complete, final video:

(define (talk->video talk)
  "Given a talk, return a complete video, with opening and closing."
  (define file-name
    (string-append (canonicalize-string (talk-speaker talk))
                   ".webm"))

  (let ((raw (ges->webm (video-ges-project (talk-bbb-data talk)
                                           (talk-start talk)
                                           (talk-end talk)
                                           #:webcam-size
                                           (talk-webcam-size talk))
                        file-name))
        (opening (opening (talk-title talk) (talk-speaker talk))))
    (video-with-opening/closing file-name raw
                                opening closing.webm)))

The very last bit iterates over the talks and returns a manifest containing all the final videos. Now we can build the ready-to-be-published videos, all at once:

$ guix build -m render-videos.scm
[… time passes…]
/gnu/store/…-emmanuel-agullo.webm
/gnu/store/…-francois-rue.webm
…

Voilà!

Image of an old TV screen showing a video opening.

Why all the fuss?

OK, maybe you’re thinking “this is just another hackish script to fiddle with videos”, and that’s right! It’s also worth mentioning another approach: Racket’s video language, which is designed to manipulate video abstractions, similar to GES but with a sweet high-level functional interface.

But look, this one’s different: it’s self-contained, it’s reproducible, and it has the right abstraction level. Self-contained is a big thing; it means you can run it and it knows what software to deploy, what environment variables to set, and so on, for each step of the pipeline. Granted, it could be simplified with appropriate high-level interfaces in Guix. But remember: the alternative is a makefile (“deployment-unaware”) completed by a README file giving a vague idea of the dependencies needed. The reproducible bit is pretty nice too (especially for a workshop on reproducibility). It also means there’s caching: videos or intermediate byproducts already in the store don’t need to be recomputed. Last, we have access to a general-purpose programming language where we can build abstractions, such as the <talk> data type, that makes the whole thing more pleasant to work with and more maintainable.

Hopefully that’ll inspire you to have a reproducible video pipeline for your next on-line event, or maybe that’ll inspire you to replace your old makefile and shelly habits for data processing!

High-performance computing (HPC) people might be wondering how to go from here and build “computing-resource-aware” or “storage-resource-aware” pipelines where each computing step could be submitted to the job scheduler of an HPC cluster and use distributed file systems for intermediate results rather than /gnu/store. If you’re one of these folks, do take a look at how the Guix Workflow Language addresses these issues.

Acknowledgments

Thanks to Konrad Hinsen for valuable feedback on an earlier draft.

About GNU Guix

GNU Guix is a transactional package manager and an advanced distribution of the GNU system that respects user freedom. Guix can be used on top of any system running the Hurd or the Linux kernel, or it can be used as a standalone operating system distribution for i686, x86_64, ARMv7, AArch64 and POWER9 machines.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. When used as a standalone GNU/Linux distribution, Guix offers a declarative, stateless approach to operating system configuration management. Guix is highly customizable and hackable through Guile programming interfaces and extensions to the Scheme language.

Urgent: zero emission transit policy [Richard Stallman's Political Notes]

US citizens: call on Congress to fully fund a zero emission transit policy now.

Urgent: lift sanctions [Richard Stallman's Political Notes]

US citizens: call on Biden to lift sanctions that are blocking Covid relief.

Dam on the Blue Nile [Richard Stallman's Political Notes]

Ethiopia has build a large dam on the Blue Nile. Egypt and Sudan worry that by retaining water in Ethiopia, it will reduce their share of the water.

Egypt and Sudan have coasts, and can do solar-powered desalinization (very expensive) to purify sea water. That is the only solution for the medium-term — the only way to make enough potable water for the growing population.

In the long term, curbing and then reversing population growth is the only way.

Reopened too fast [Richard Stallman's Political Notes]

The UK has reopened too fast; the Delta variant of Covid-19 is spreading. This was quite predictable.

What the UK needs is leadership in taking the steps to prevent transmission. But it won't get that from Bogus Johnson, whose repeated policy changes have not given Britons a reason to trust his judgment. Officials' breaking their own hygiene rules have led Britons to disregard the same rules.

Pipeline [Richard Stallman's Political Notes]

The Keystone XL pipeline has been cancelled. The owner yielded in the face of Biden's decision to revoke the permit.

Now what about the Line 3 pipeline? That would carry tar sands oil too, if it is allowed to be built.

1836 Project [Richard Stallman's Political Notes]

Texas Republicans have launched the "1836 Project" to teach a distorted version of the reason for Texas's secession from Mexico.

I expect that the Mexican government and Mexicans elites found the rapid arrival of so many border-crossing immigrants in Texas worrisome in itself, independent of the fact that some of them illegally had slaves.

I suspect that one of the aims of this project is to pave the way for an attempt to take nondemocratic control of the US government, or else secede.

Feeds

FeedRSSLast fetchedNext fetched after
XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
a bag of four grapes XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
A Smart Bear: Startups and Marketing for Geeks XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
Anarcho's blog XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
Ansible XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
Bad Science XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
Black Doggerel XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
Blog – Official site of Stephen Fry XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
Broodhollow XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
Charlie Brooker | The Guardian XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
Charlie's Diary XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
Chasing the Sunset - Comics Only XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
Clay Shirky XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
Coding Horror XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
Cory Doctorow – Boing Boing XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
Cory Doctorow's craphound.com XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
Ctrl+Alt+Del Comic XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
Cyberunions XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
David Mitchell | The Guardian XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
Debian GNU/Linux System Administration Resources XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
Deeplinks XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
Diesel Sweeties webcomic by rstevens XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
Dilbert XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
Dork Tower XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
Economics from the Top Down XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
Edmund Finney's Quest to Find the Meaning of Life XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
Eerie Cuties XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
EFF Action Center XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
Enspiral Tales - Medium XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
Erin Dies Alone XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
Events XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
Falkvinge on Liberty XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
Flipside XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
Flipside XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
Free software jobs XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
Full Frontal Nerdity by Aaron Williams XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
General Protection Fault: The Comic Strip XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
George Monbiot XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
Girl Genius XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
God Hates Astronauts XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
Graeme Smith XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
Groklaw XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
Grrl Power XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
Hackney Anarchist Group XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
http://calendar.google.com/calendar/feeds/q7s5o02sj8hcam52hutbcofoo4%40group.calendar.google.com/public/basic XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
http://dynamic.boingboing.net/cgi-bin/mt/mt-cp.cgi?__mode=feed&_type=posts&blog_id=1&id=1 XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
http://eng.anarchoblogs.org/feed/atom/ XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
http://feed43.com/3874015735218037.xml XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
http://feeds2.feedburner.com/GeekEtiquette?format=xml XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
http://fulltextrssfeed.com/ XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
http://london.indymedia.org/articles.rss XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
http://pipes.yahoo.com/pipes/pipe.run?_id=ad0530218c055aa302f7e0e84d5d6515&amp;_render=rss XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
http://the-programmers-stone.com/feed/ XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
http://thecommune.co.uk/feed/ XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
http://www.airshipentertainment.com/buck/buckcomic/buck.rss XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
http://www.airshipentertainment.com/growf/growfcomic/growf.rss XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
http://www.airshipentertainment.com/myth/mythcomic/myth.rss XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
http://www.baen.com/baenebooks XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
http://www.dcscience.net/feed/medium.co XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
http://www.feedsapi.com/makefulltextfeed.php?url=http%3A%2F%2Fwww.somethingpositive.net%2Fsp.xml&what=auto&key=&max=7&links=preserve&exc=&privacy=I+accept XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
http://www.freedompress.org.uk/news/feed/ XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
http://www.steampunkmagazine.com/inside/feed/ XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
http://www.tinycat.co.uk/feed/ XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
https://hackbloc.org/rss.xml XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
https://kajafoglio.livejournal.com/data/atom/ XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
https://philfoglio.livejournal.com/data/atom/ XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
https://studiofoglio.livejournal.com/data/atom/ XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
https://twitter.com/statuses/user_timeline/22724360.rss XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
https://web.randi.org/?format=feed&type=rss XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
https://www.DropCatch.com/domain/ubuntuweblogs.org XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
https://www.goblinscomic.com/category/comics/feed/ XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
https://www.hackneysolidarity.info/rss.xml XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
https://www.newstatesman.com/feeds/blogs/laurie-penny.rss XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
https://www.patreon.com/graveyardgreg/posts/comic.rss XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
https://www.rightmove.co.uk/rss/property-for-sale/find.html?locationIdentifier=REGION^876&maxPrice=240000&minBedrooms=2&displayPropertyType=houses&oldDisplayPropertyType=houses&primaryDisplayPropertyType=houses&oldPrimaryDisplayPropertyType=houses&numberOfPropertiesPerPage=24 XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
Humble Bundle Blog XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
I, Cringely XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
Irregular Webcomic! XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
Joel on Software XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
Judith Proctor's Journal XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
Krebs on Security XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
Lambda the Ultimate - Programming Languages Weblog XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
LLVM Project Blog XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
Looking For Group XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
Loomio Blog XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
LWN.net XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
Menage a 3 XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
Mimi and Eunice XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
Neil Gaiman's Journal XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
Nina Paley XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
O Abnormal – Scifi/Fantasy Artist XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
Oglaf! -- Comics. Often dirty. XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
Oh Joy Sex Toy XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
Order of the Stick XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
Original Fiction – Tor.com XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
OSnews XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
Paul Graham: Unofficial RSS Feed XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
Penny Arcade XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
Penny Red XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
PHD Comics XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
Phil's blog XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
Planet Debian XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
Planet GNU XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
Planet GridPP XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
Planet Lisp XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
Pluralistic: Daily links from Cory Doctorow XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
Property is Theft! XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
PS238 by Aaron Williams XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
QC RSS XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
Radar XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
RevK®'s ramblings XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
Richard Stallman's Political Notes XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
Scenes From A Multiverse XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
Schneier on Security XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
SCHNEWS.ORG.UK XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
Scripting News XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
Seth's Blog XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
Skin Horse XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
Spinnerette XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
Starslip by Kris Straub XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
Tales From the Riverbank XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
The Adventures of Dr. McNinja XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
The Bumpycat sat on the mat XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
The Command Line XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
The Daily WTF XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
The Monochrome Mob XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
The Non-Adventures of Wonderella XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
The Old New Thing XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
The Open Source Grid Engine Blog XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
The Phoenix Requiem XML 15:49, Tuesday, 15 June 16:29, Tuesday, 15 June
The Rogues Gallery XML 15:49, Tuesday, 15 June 16:37, Tuesday, 15 June
The Stranger, Seattle's Only Newspaper: Savage Love XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
TorrentFreak XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
towerhamletsalarm XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
Twokinds XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
UK Indymedia Features XML 16:28, Tuesday, 15 June 17:10, Tuesday, 15 June
Uploads from ne11y XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
Uploads from piasladic XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June
Use Sword on Monster XML 15:49, Tuesday, 15 June 16:36, Tuesday, 15 June
Wayward Sons: Legends - Sci-Fi Full Page Webcomic - Updates Daily XML 15:49, Tuesday, 15 June 16:35, Tuesday, 15 June
What If? XML 15:49, Tuesday, 15 June 16:30, Tuesday, 15 June
Whatever XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
Whitechapel Anarchist Group XML 16:07, Tuesday, 15 June 16:56, Tuesday, 15 June
WIL WHEATON dot NET XML 15:49, Tuesday, 15 June 16:33, Tuesday, 15 June
wish XML 15:49, Tuesday, 15 June 16:34, Tuesday, 15 June
xkcd.com XML 15:49, Tuesday, 15 June 16:32, Tuesday, 15 June