Use stdbuf to tee without buffering

Do you want to tee the output of a command to a file, but see it in your terminal too, without buffering? The stdbuf command can do this for you:

$ sudo stdbuf --output=L tcpdump -i any -tttt -n 'udp port 5353' | tee -a tcpdump-mdns
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
2023-06-15 11:55:16.637670 eth0 M   IP 10.0.0.23.5353 > 224.0.0.251.5353: 0 A (QM)? winnebago.local. (28)
2023-06-15 11:55:16.744660 eth0 M   IP 10.0.0.42.5353 > 224.0.0.251.5353: 0*- [0q] 1/0/0 (Cache flush) A 10.0.0.42 (38)
...

Which uninstalled package provides a file?

$ apt-file find guestmount
guestmount: /usr/bin/guestmount
guestmount: /usr/share/bash-completion/completions/guestmount
guestmount: /usr/share/doc/guestmount/changelog.Debian.gz
guestmount: /usr/share/doc/guestmount/copyright
guestmount: /usr/share/man/ja/man1/guestmount.1.gz
guestmount: /usr/share/man/man1/guestmount.1.gz
guestmount: /usr/share/man/uk/man1/guestmount.1.gz

Scripting DNS lookups

Are you writing a script and some command doesn’t accept hostnames and you don’t want to inline the IP address? dig +short is your friend!

$ dig +short gbenson.net
69.163.152.201

Which APT repository did a package come from?

$ apt policy wget
wget:
  Installed: 1.21.2-2ubuntu1
  Candidate: 1.21.2-2ubuntu1
  Version table:
 *** 1.21.2-2ubuntu1 500
        500 http://gb.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
        100 /var/lib/dpkg/status

Python debugger

By the way, if you’ve never used Python’s debugger, it really is as simple as adding a call to the built-in function breakpoint() at the point you want it to stop.

Flask on Elastic Beanstalk

I had a play with Elastic Beanstalk the other day. It’s one of those things people turn their noses up at, but it seems pretty good for prototyping and small things. My biggest issue so far has been that, for Python applications, it expects a WSGI callable called application in the file application.py… but I was using Flask, and every single Flask application ever has a WSGI callable called app in the file app.py. I tried to not care, but it got too much after about an hour so I went and found how to override it:

$ cat .ebextensions/01_wsgi.config
option_settings:
  aws:elasticbeanstalk:container:python:
    WSGIPath: "app:app"

Thank you Nik Tomazic for that! (=⌒‿‿⌒=)

The other AWS IAM SSO problem

The other thing you’ll run into using IAM users in AWS CLI is that a lot of things don’t support SSO sessions anyway. If you configure an IAM user with an SSO session name as recommended you’ll get errors like this:

$ eb init -p python-3.8 eb-flask-app
ERROR: InvalidConfigError - The profile "default" is configured to use SSO but is missing required configuration: sso_start_url, sso_region

and this:

$ terraform apply
| Error: configuring Terraform AWS Provider: loading configuration: profile "default" is configured to use SSO but is missing required configuration: sso_region, sso_start_url

You can fix these by configuring without an SSO session:

$ aws configure sso
SSO session name (Recommended): 
WARNING: Configuring using legacy format (e.g. without an SSO session).
Consider re-running "configure sso" command and providing a session name.
SSO start URL [None]: https://whatever.awsapps.com/start
SSO region [None]: us-east-1
 ...

You can also fix them by just editing your ~/.aws/config, and copying the sso_start_url and sso_region keys from the [sso-session ...] section into the relevant user’s section, but that might be a hack too far!

Using AWS CLI with IAM users

When you configure AWS CLI to use an IAM user, the first thing it asks for is an SSO session name. Don’t put whitespace or punctuation in it. The command doesn’t tell you this, but it’s going to use what you enter as an identifier, and fail with a cryptic error:

$ aws configure sso
SSO session name (Recommended): AWS CLI on Gary's Chromebook
SSO start URL [None]: https://whatever.awsapps.com/start
SSO region [None]: us-east-1
SSO registration scopes [sso:account:access]:

An error occurred (InvalidClientMetadataException) when calling the RegisterClient operation:

You need to put something like “gbenson” in there.

sed trick

I discovered a new sed trick today:

sed -i~

“I” stands for “in place”. It edits the files in place! And makes a backup if you want!

Container debugging minihint

What’s in my container?

$ podman ps --ns
CONTAINER ID  NAMES            PID    CGROUPNS  IPC         MNT         NET         PIDNS       USERNS      UTS
fe11359293e8  eloquent_austin  11090            4026532623  4026532621  4026532421  4026532624  4026531837  4026532622

$ sudo ls -l /proc/11090/root/
total 22628
lrwxrwxrwx.   1 root root        7 Jul 25  2019 bin -> usr/bin
dr-xr-xr-x.   2 root root        6 Jul 25  2019 boot
drwxr-xr-x.   5 root root      360 Jan 24 12:03 dev
drwxr-xr-x.   1 root root      183 Jan 23 16:43 etc
 ...

Thank you.
[28 Jan@1135UTC] UPDATE—Actually, this doesn’t seem to work with newer systems, sorry!