notes by alifeee profile picture tagged scripting (5) rss

return to notes / blog / website / weeknotes / linktree

here I may post some short, text-only notes, mostly about programming. source code.

tags: all (19) scripting (5) bash (4) linux (3) html (2) markdown (2) obsidian (2) shortcuts (2) ActivityPub (1) advent-of-code (1) ............ see all (+24)

turning my clipboard into a blockquote on Linux # prev single next top

tags: scripting, linux, markdown • 237 'words', 71 secs @ 200wpm

I like markdown. I use Obsidian a lot, and write a lot in GitHub issues. Something useful I usually do is quote other people's words, so in markdown it would look like:

The Met office said

> it will definitely snow tonight
>
> like... 100%

I found that I can use a command xclip to get/set my clipboard on Linux, and I use a lot of sed to do word replacement, so I realised I could copy the text

it will definitely snow tonight

like... 100%

and then run this command in my terminal (xclip gets/sets the clipboard, sed replaces ^ (the start of each line) with > )

xclip -selection c -o | sed "s/^/> /" | xclip -selection c

which would get my clipboard, replace the start of each line with a quote, and set the clipboard, setting the clipboard to:

it will definitely snow tonight

like... 100%

I've set aliases for these commands so I can use them quickly in my terminal as:

alias getclip='xclip -selection c -o'
alias setclip='xclip -selection c'
alias quote='getclip | sed "s/^/> /" | setclip'

but also I created a keyboard shortcut in Gnome, CTRL + SUPER + Q, which will quote my clipboard. I had to set the shortcut to run bash -c 'xclip -selection c -o | sed "s/^/> /" | xclip -selection c' as I don't think pipes sit well in shortcuts.

But now I can really easily...

quooooooooote

back to top

getting latlong coordinates from an address with geocode.xyz # prev single next top

tags: scripting, maps • 369 'words', 111 secs @ 200wpm

I like maps. I make maps. Mostly from worse maps or data that is not in map form. See some of mine on https://alifeee.co.uk/maps/.

One thing I've been doing for a map recently is geocoding, which is turning an address (e.g., "Showroom Cinema, Paternoster Row, Sheffield") into latitude/longitude coordinates.

https://geocode.xyz/ provides a free geocoding API on https://geocode.xyz/api which is rate limited to one request per second.

Here is a script to wrap that API for using it as a script. It's not amazing but it works.

#!/bin/bash

loc="${1}"
throttled=1

while [ $throttled = 1 ]; do
  resp=$(curl -s -X POST -d locate="${loc}" -d geoit="json" "https://geocode.xyz")
  if [[ "${resp}" =~ Throttled ]]; then
    echo "throttled... retrying..." >> /dev/stderr
    throttled=1
  else
    throttled=0
  fi
  sleep 1
done

echo "got response: ${resp}" >> /dev/stderr

json=$(echo "${resp}" | jq | sed 's/ {}/""/g')

basic=$(echo "${json}" | jq -r '
.latt + "\t" +
.longt + "\t" +
.standard.confidence + "\t"'
)

standard=$(echo "${json}" | jq -r '
.standard.addresst? + "\t" +
.standard.statename? + "\t" +
.standard.city? + "\t" +
.standard.prov? + "\t" +
.standard.countryname? + "\t" +
.standard.postal? + "\t"
')

alt=$(echo "${json}" | jq -r '
.alt?.loc?.addresst + "\t" +
.alt?.loc?.statename + "\t" +
.alt?.loc?.city + "\t" +
.alt?.loc?.prov + "\t" +
.alt?.loc?.countryname + "\t" +
.alt?.loc?.postal + "\t"
')
echo "${basic}${standard}${alt}" | sed '1s/^/latitude\tlongitude\tconfidence\taddress\tstate\tcity\tprovince\tcountry\tpost code\talt address\talt state\talt city\talt province\talt country\talt postal\n/'

and then you can use it like

$ ./geocode.sh "Showroom Cinema, Paternoster Row, Sheffield"
throttled... retrying...
throttled... retrying...
got response: {   "standard" : {      "stnumber" : "1",      "addresst" : "Paternoster Row",      "statename" : "England",      "postal" : "S1",      "region" : "England",      "prov" : "UK",      "city" : "Sheffield",      "countryname" : "United Kingdom",      "confidence" : "0.9"   },   "longt" : "-1.46544",   "alt" : {},   "elevation" : {},   "latt" : "53.37756"}
latitude	longitude	confidence	address	state	city	province	country	post code	alt address	alt state	alt city	alt province	alt country	alt postal
53.37756	-1.46544	0.9	Paternoster Row	England	Sheffield	UK	United Kingdom	S1

The results are "ok". They're pretty good for street addresses, but I can see a lot of wrong results. I might try and use another API like OpenStreetMap's or (shudders) Google's.

back to top

updating a file in a GitHub repository with a workflow # prev single next top

tags: github-actions, github, scripting • 196 'words', 59 secs @ 200wpm

often, I want to keep a file in a repository up to date using a script.

here is the most simple example of a workflow that does that, which could be placed in, e.g., .github/workflows/update.yml

name: update stock.txt

on:
  push:
    branches: ["main"]

permissions:
  contents: write

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: update stock.txt
        run: ./scripts/compute_stock.sh stocktaking.csv > stock.txt

      - name: Commit changes
        uses: stefanzweifel/git-auto-commit-action@v5
        with:
          commit_message: updated stock.txt with latest numbers

This workflow was created for https://github.com/lipu-tenpo/print-and-post to keep a file listing stock up to date with the "logs" of when I acquired and un-acquired stuff.

I initially thought about doing this using Git Hooks, but I want to be able to edit the "logs" from my phone or via a browser, where I wouldn't be able to trigger the git hook.

So, I'm doing it in a proprietary "GitHubby" way, which would be annoying to change if I changed to, say, GitLab. But, here we are. Technology lock-in is real.

back to top

linting markdown from inside Obsidian # prev single next top

tags: obsidian, scripting, markdown • 382 'words', 115 secs @ 200wpm

I like Obsidian. I started using it recently instead of Notion. It is very nice.

The fact that it is static files is nice. The fact that those files are markdown is even nicer.

I sync it to all my devices with Syncthing. Sometimes there are sync conflicts, but https://github.com/friebetill/obsidian-file-diff makes solving that super easy.

Anyway, I've been writing these notes in Obsidian. I have then been copying and pasting the content into https://dlaa.me/markdownlint/ to find problems with my Markdown formatting. It's mainly when I forget to wrap links in <> as this makes them not render as HTML links - I sort of like this as you (my automatic tool) shouldn't try and decide what is and isn't a link, but also maybe you should because you can probably recognise them pretty well with a very established regex by now.

Anyway, I found an Obsidian extension which lets you specify shell commands https://github.com/Taitava/obsidian-shellcommands that you can run via the command palette. This seems super neat - I can do ANYTHING now.

Anyway, I installed it and made a command to lint the current markdown file. I had to install npm globally because it wasn't working when being called from the Obsidian script, and then I made this command to run the lint.

First install the markdownlint CLI:

npm install -g markdownlint-cli

The command is:

(cd {{folder_path:absolute}}; source /usr/alifeee/.nvm/nvm.sh && nvm use 20 && markdownlint {{file_name}} --disable MD013 && echo "no lint issues!")

I disabled MD013 which is the insane rule which wants you to have loads of line breaks per paragraph (I prefer my paragraph to be one really really long line please).

It's not perfect (the output is just in an ugly pop up window), but it is nice to run it locally.

Next... perhaps automatic uploading of notes from Obsidian, and I won't even have to open https://github.com/alifeee/blog/tree/main/notes to add a note to this site... dangerous...

(p.s., I managed to write this without any lint issues ;] )

back to top

installing nvm globally so automated scripts can use node and npm # prev single next top

tags: node, scripting • 366 'words', 110 secs @ 200wpm

I like to make things automatic, and happen without me being there.

I also like to use things people have made using Node and npm.

I also like to use nvm to manage Node versions.

Here is the trouble: using the npm command or using command line scripts installed globally via npm install -g ... as a user that is not you.

I am alifeee. I would like other users (e.g., www-data) to be able to use npm, so that I can, say, make a CGI script that changes a file, and then runs npm run build. I do this exact thing for https://github.com/alifeee/simple-calendar, which uses yaml files and an Eleventy website to make a simple calendar. Another one is that I want to use npm commands in scripts run with https://github.com/Taitava/obsidian-shellcommands, which does not run as my current user.

The problem is that the normal way to install nvm installs it into your user folder (i.e., /home/alifeee), so other users can't use it.

It took me way too long to figure this out (banging my head against npm-shaped walls for hours), but I have switched from running the default install script on https://github.com/nvm-sh/nvm to now doing this:

## remove existing nvm/npm installation
rm -rf ~/.nvm
rm -rf ~/.npm
nano ~/.bashrc # (remove nvm sections)
# install nvm to folder
mkdir -p /usr/alifeee/.nvm
export XDG_CONFIG_HOME="/usr/alifeee"
export NVM_DIR=/usr/alifeee/.nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash
# install npm version you want
nvm install 20
npm use 20

Then, I can use nvm because it's sourced in my ~/.bashrc (as before), but importantly, any user can use npm and Node by running:

## safe (same way it's done in .bashrc - check if the file exists)
export NVM_DIR="/usr/alifeee/.nvm" && [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"; nvm use 20; npm --version
## less 'safe' but works fine
source /usr/alifeee/.nvm/nvm.sh && nvm use 20 && npm -v

And now, I am free.

back to top