Terminal Fun: Basic Utils Part 3

Jonatan Ivanov
Dev

This is the third post of a series where I want to give some tips and tricks for the Terminal/CLI.
You might want to check out the first and the second posts. As previously, buckle-up, put your helmet on, grab a … frying pan and have fun.

Network Tools

httpie

httpie is a command-line HTTP client (like curl or wget) for the API era. It comes with JSON support, syntax highlighting, persistent sessions, wget-like downloads, plugins, and more. See the features on the Official Site or the readme.

httpie-demo

httpstat

httpstat visualizes curl statistics. It tells you where did your request spend time.

httpstat-demo

sslscan

sslscan is a test utility that tells you information about SSL/TLS enabled services and discovers supported cipher suites. Check the readme.

sslscan-demo

mitmproxy

mitmproxy is your swiss-army knife for debugging, testing, privacy measurements, and penetration testing. It can be used to intercept, inspect, modify, and replay web traffic such as HTTP/1, HTTP/2, WebSockets, or any other SSL/TLS-protected protocols.

mitmproxy-demo

mtr

mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.

As mtr starts, it investigates the network connection between the host mtr runs on and a user-specified destination host. After it determines the address of each network hop between the machines, it sends a sequence ICMP ECHO requests to each one to determine the quality of the link to each machine. As it does this, it prints running statistics about each machine.


Others

tldr

tldr shows you help pages:

tldr-demo

navi is like an interactive cheatsheet:

navi-demo

up

up stands for Ultimate Plumber, it’s a tool for writing Linux pipes interactively, with a live preview of command results.

It helps interactively and incrementally explore textual data by making it easier to quickly build complex pipelines, thanks to a fast feedback loop. This is achieved by boosting any typical text-processing utils such as grep, sort, cut, paste, awk, wc, perl, etc. by providing a quick, interactive, scrollable preview of their results.

up-demo You can watch the same as an asciicast.

hhighlighter

(hhighlighter](https://github.com/paoloantinori/hhighlighter) (h) is a tiny utility to highlight multiple keywords with different colors, it’s pretty nice for logs.

Pro Tip: alias hl='h ERROR INFO WARN DEBUG' (hl as highlight logs)
and you can do: cat logs/service.log | hl

asciicast

mainframer

mainframer was created to allow you to move your build process from a local machine to a remote one. Since it is a nice combo of ssh + rsync, it supports anything you can execute as a command. It will sync files to a remote machine, execute a command, and sync files back.

I found it very handy for performance testing, where the test suite is on your local box which will be copied (rsync) to a remote machine, then the tests will be executed (ssh) and the report will be synced back (rsync). It’s very simple solutions to run heavy operations on remote machines.

Please notice that 3.x is a complete rewrite of the project and it is not ready yet, the current version that you can use is 2.x.

mainframer-demo

GNU datamash

GNU datamash is a command-line program which performs basic numeric, textual and statistical operations on input textual data files (e.g.: count, sum, min, max, mean, stdev, string coalescing).

Example: sum the values in the first column

$ seq 10 | datamash sum 1
55

Example: calculating basic statistics like mean, 1st quartile, median, 3rd quartile, IQR, sample-standard-deviation, and p-value of Jarque-Bera test for normal distribution:

$ datamash -H mean 1 q1 1 median 1 q3 1 iqr 1 sstdev 1 jarque 1 < FILE.TXT
mean(x)   q1(x)  median(x)  q3(x)   iqr(x)  sstdev(x)  jarque(x)
45.32     23     37         61.5    38.5    30.4487    8.0113e-09

Thanks for the tip to Daniel Hinojosa

asciinema

You might have noticed that some of your demos above were recorded in a seemingly weird but pretty nice way. With asciinema you can record and share your terminal sessions, the right way. :) You can forget screen recording apps and blurry videos. It’s a lightweight, purely text-based approach to terminal recording.

asciicast

Posts in this series

Thanks to Peter, who let me know about some of these tools.