Command Line

Planted 02021-05-28

First, I’m currently running an Intel Mac, and using Homebrew as a package manager.

References

• Taco Bell Programming
• Homebrew analytics
• Missing Semester of Your CS Education
• Unofficial Guide to dotfiles on GitHub
• Wes Bos Command Line Power User
• Command-line Tools can be 235x Faster than your Hadoop Cluster
• Cascadia 2011 - The Unix Chainsaw by Gary Bernhardt

Git

So You Think You Know Git - FOSDEM 2024

git status
git add <file>
git commit -m ""
git pull origin main
git push origin main

Git in two minutes

Config

• /scripts
• /dotfiles

To maintain configurations across computers I keep a /dotfiles folder in Github. This folder has the .zshrc file that I use. (might be better as a Gist?)

I clone the /dotfiles folder to the Home directory.

ZSH Config

You can set $ZDOTDIR in .zshenv to specify an alternative location for the rest of your zsh configuration.

export ZDOTDIR="$HOME/dotfiles"

https://unix.stackexchange.com/a/487889

• .zshenv: Read every time
• .zprofile: Read at login
• .zshrc: Read when interactive
• .zlogin: Read at login, but after .zshrc
• .zlogout: Read at logout (in login shell)

Scripts // alias

# Lists all processes running on a given tcp port
function listport {
    local port=${1:?"The port must be specified."}
    local processes=$(lsof -i tcp:"$port" | grep LISTEN)
    if [[ -z $processes ]]; then
        echo "No processes running on tcp:$port"
    else
        echo "$processes" | awk '{print $1, $2}'
        printf "Do you want to kill the process? (y/n) "
        read answer
        if [[ $answer == "y" ]]; then
            echo "$processes" | awk '{print $2}' | xargs kill
        fi
    fi
}

# Get CPU usage for Visual Studio Code extensions
function codeplease {
    ps aux | awk 'NR==1 { print "CPU", "Extension" }\
    NR>1 {\
        if ($0 ~ /Code\.app\/Contents\/Resources\/app\/extensions/) {\
            match($0, /Code\.app\/Contents\/Resources\/app\/extensions[^[:space:]]+/);\
            print $3, substr($0, RSTART, RLENGTH)\
        }\
        else
        if ($0 ~ /\.vscode\/extensions/) {\
            match($0, /\.vscode\/extensions[^[:space:]]+/);\
            print $3, substr($0, RSTART, RLENGTH)\
        }\
    }'
}

• browse: /scripts/startchrome.sh
• update: /scripts/update.sh
  • update brew and git pull /scripts and /dotfiles
• post: /Lukasmurdock.github.io/post.sh
  • create new post file and open in VS Code

Utilities

• grep
• sed
• wget
• curl
• awk (gawk)
• FFmpeg
• youtube-dl
• ImageMagick
• taskwarrior
• jq
• autojump
• httrack
• tree
• tmux
• htop
• carbon-now
• github-cli
• ghostscript
• comm

Handy NPM Packages

• (Manage NPM with volta (volta install npm@7))
• Release: automatically generates a new GitHub Release and populates it with the changes (commits) made since the last release
• npm-check-updates: ncu -i
• npkill
• npm-home
• carbon-now-cli
  • carbon-now <file> -l $HOME/documents/carbon-images
  • carbon-now -l $HOME/documents/carbon-images --from-clipboard

Awesome lists:

• Awesome CLI apps
• Awesome command line apps
• Awesome macOS command line

Basic Terminal Things

Keyboard shortcuts in Terminal on Mac

• Clear terminal: Command + K
• Clear terminal line: Control + U
• Delete forward to the end of the word: Escape + D
  • Option + D is available with Use Option as Meta Key)
• Reposition insertion point: Option + Click
• Open an app: open -a slack
• Who is listening on a given TCP port on Mac OS X sudo lsof -iTCP -sTCP:LISTEN -n -P

iTerm2

iTerm2 Features

• Autocomplete: Command + ;

iTerm2 Cheatsheet

Turn on Natural Text Editing

• Have left option key act as escape

pipes |

The Pipe character | is used to combine two or more commands, and in this, the output of one command acts as input to another command

cat

cat is mostly used for displaying, combining and creating text files.

grep

Grep lets you search files for text that matches a pattern.

Wizard zines on grep

sed

Sed is mostly used for replacing text in a file.

Wizard Zine about sed

You can replace day with night in a file with: sed 's/day/night/' sample.txt

The sed substitute command has four parts:

• Substitute command: s
• Delimiter: /../../
• Regular Expression Pattern Search Pattern: day
• Replacement string: night

An introduction and tutorial to Sed

Flags

• sed -i ""

Pattern Flags

• /../../g Global replacement

Use sed on inline string

echo "string" | sed 's/ring/yle/'
# // style

Delete the first instance of a specific character

Delete an ‘a’ from a file: sed 's/a//' sample.txt

Delete all instances of a specific character

Delete every ‘a’ from a file: sed 's/a//g' sample.txt

Delete or substituting string

Delete “windows” with “Mac”: sed 's/Mac/windows//' sample.txt

Regex

Delete first character in every line: sed 's/^.//' sample.txt

Delete last character in every line: sed 's/.$//' sample.txt

Delete the first and last character in every line: sed 's/.//;s/.$//' sample.txt

Delete everything in a line followed by a character: sed 's/a.*//' sample.txt

ps

ps shows what processes are running. You can use the ps command to find the process identifier (PID) of a process.

You can use the PID to kill the process

Wizard zine for ps

Awk, sed, grep

Package things (Homebrew)

First, make sure you have Homebrew installed.

Brewfile tips

Create a Brewfile

brew bundle dump --file=~/dotfiles/Brewfile --force

wget

wget is a package for retrieving files with the most widely used Internet protocols (HTTP, HTTPS, FTP and FTPS). It’s strong point is recursive downloads.

Spider mode

wget --spider -r https://example.com/ 2>&1 | tee crawl.txt

Get unique urls from crawl

grep -Eo 'https?://[^ ]+' crawl.txt | sort -u > unique_urls.txt

Get unique folders from crawl (the sed part removes trailing slashes to deduplicate)

grep -Eo 'https?://[^ ]+' crawl.txt | grep -Eo '(https?://[^/]+/[^ ]+)' | sed 's/\/$//' | sort -u > unique_folders.txt

curl

curl is a library for transferring data with URLs. It’s great for testing APIs.

Wizard zines for curl

Flags

• Show response headers: -i
• Show only response readers: -I
• Make a POST request: -X POST
• Add a header: -H "Content-Type: application/json"
• POST data: --data '{"name": "Lukas"}'
• Follow 3xx redirects: -L
• Show request headers and stuff: -v

awk (gawk)

AWK is a programming language for manipulating columns of data. gawk is the GNU implementation of awk with extensions.

Wizard Zines on awk

FFmpeg

FFmpeg is a package to record, convert and stream audio and video.

ffmpeg buddy

# Convert .mp4 to .mp3
ffmpeg -i FILE-TO-CONVERT.mp4 -q:a 0 -map a OUTPUT-FILE-NAME.mp3

# Convert .mov to .mp4
ffmpeg -i INPUT-VIDEO.mov -vcodec h264 -acodec mp2 OUTPUT-VIDEO.mp4

# Combine video files
ffmpeg -safe 0 -f concat -i fileCombine.txt -c copy output.mp4
## fileCombine.txt
### file ./video01.mov
### file ./video02.mov

See:

• Combine video files
• Normalize audio

youtube-dl

youtube-dl is a package to download videos from YouTube and a few more sites.

# Download video from YouTube
youtube-dl --recode-video mp4 VIDEO-LINK-OR-PLAYLIST-ID

# Download audio from YouTube
youtube-dl -f bestaudio --extract-audio --audio-format mp3 --audio-quality 0 VIDEO-LINK-OR-PLAYLIST-ID

# Download single Frame image from YouTube
ffmpeg -ss "[SCREENSHOT_TIME]" -i $(youtube-dl -f 22 --get-url "[YOUTUBE_URL]") -vframes 1 -q:v 2 "[FILENAME].jpeg"

(H/T)

ImageMagick

ImageMagick is a package to create, edit, compose, or convert images.

Fred’s ImageMagick Scripts

# Convert image (webp to jpg)
magick INPUT.webp OUTPUT.jpg

# Bulk image convert (web to jpg)
magick mogrify -format jpg *.webp

# JPG compression
convert INPUT.jpg -sampling-factor 4:2:0 -strip -quality 85 -interlace JPEG -colorspace RGB OUTPUT.jpg

Docker

Docker delivers software by isolated containers.

# Get Jekyll in docker or something
docker run --rm --volume="$PWD:/srv/jekyll" -p 4000:4000 jekyll/jekyll jekyll serve

Docker-compose up

Taskwarrior

Taskwarrior manages a TODO list from the command line.

Taskwarrior examples

Taskwarrior best practices

# Add a project and assign it a project
task add project:{PROJECT} {INSERT TASK}

# Add a due date where appropriate:
task {ID} modify due:31st

# When you start working on a task, mark it started
task {ID} start

# If you know the priority of the task
task {ID} modify priority:{L,M,H}

# Add useful tags:
task {ID} modify +{TAG} +{TAG}

# Urgent next task?
task {ID} modify +next

# Represent dependencies where appropriate
task {ID} modify depends:{OTHER_ID}

Taskwarrior date synonyms

jq

jq is a command-line JSON processor.

autojump

autojump is a faster way to navigate your filesystem.

tree

htop

tmux

Idk bouta figure this out though.

Tmux cheatsheet

Start tmux in iTerm2: tmux -CC

Carbon-now

Github CLI

GitHub CLI brings GitHub to your terminal.

Github CLI command list

Get notifications: gh api notifications

Only get the info we want: gh api notifications | jq '.[] | {title: .subject.title, repository: .repository.name}'

Ghostscript

Ghostscript is an interpreter for the PostScript® language and PDF files.

# Combine PDFs into one
gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=merged.pdf file1.pdf file2.pdf

# Combine all files current directory into a file outside the directory
gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=../merged.pdf $(echo $(ls))

macOS command-line tools you might not know about