Table operations like in Nu shell in BASH

By meain/blog November 16, 2019 · Edited February 6, 2026

Hi, another installment of why people should just write more bash. I am not sure if you are aware of it. There is a new shell written in Rust called Nu Shell (https://www.nushell.sh/). One of the

Hi, another installment of why people should just write more bash. I am not sure if you are aware of it. There is a new shell written in Rust called Nu Shell (https://www.nushell.sh/). One of the main ideas of it is that you treat every data as table data and then work on top if as if you were working with SQL queries.

I am not a big fan of this approach personally. I would stick to treating data as a text stream than a tabular list. Sure, it will let you put some abstraction on top of which might make your life a bit simpler in some case, but you are missing out the power of just treating is as a simple text stream.

I was inspired to write this blog after seeing this tweet (https://twitter.com/nixcraft/status/1195665338416787456?s=09) from nixcraft. You can do a lot of stuff listed in working with tables in nushell (https://book.nushell.sh/en/working_with_tables) right in BASH.

I don’t wanna BASH(pun intended) Nu shell. A lot of people seem to love it, plus Nu helps with making items into a table in the first place easier. I just want people to understand that you don’t need Nu shell to do a lot of the things that they are mentioning.

OK, enough blabbering, lets get to the actual content.

I just decided to take a few examples that they have in their book (https://book.nushell.sh/) and show how you would achieve the same thing in BASH.

I am gonna use ls as the data input here as they seem to use it for their demo in working with tables (https://book.nushell.sh/en/working_with_tables). In ls, the data is already in columns, so I guess you could say that it already looks like a table but without borders (lets tare down the borders).

Basics # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#basics) Setting baseline - bare bones ls (not exactly) # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#setting-baseline---bare-bones-ls-(not-exactly)) This is the output from my blog directly. Its in jekyll, nothing fancy.

total 64 -rw-r–r– 1 meain staff 1077 Aug 10 17:59 LICENSE.md -rw-r–r– 1 meain staff 139 May 27 2017 README.md -rw-r–r– 1 meain staff 1605 Aug 10 18:47 _config.yml drwxr-xr-x 13 meain staff 416 Nov 13 10:56 _includes drwxr-xr-x 6 meain staff 192 Nov 12 21:43 _layouts drwxr-xr-x 26 meain staff 832 Nov 16 17:49 _posts drwxr-xr-x 12 meain staff 384 Nov 16 17:15 _site -rw-r–r– 1 meain staff 533 Aug 10 17:59 about.md drwxr-xr-x 6 meain staff 192 Mar 20 2018 assets -rwxr-xr-x 1 meain staff 2166 Nov 13 10:55 favicon.png -rw-r–r– 1 meain staff 1291 May 23 2017 feed.xml -rw-r–r– 1 meain staff 863 Aug 27 2017 index.html -rwxr-xr-x 1 meain staff 13 Oct 11 20:16 run-dev

Actually we are going to use ls | tail -n+2 from here on out as there is that total 64 thingy on top and we need to get rid of that when working with “table” ish data.

You might know about tail already but if you pass in a value like tail -n+ it will actually show everything except that many lines-1. So with that, my output will look something like this.

-rw-r–r– 1 meain staff 1077 Aug 10 17:59 LICENSE.md -rw-r–r– 1 meain staff 139 May 27 2017 README.md -rw-r–r– 1 meain staff 1605 Aug 10 18:47 _config.yml drwxr-xr-x 13 meain staff 416 Nov 13 10:56 _includes drwxr-xr-x 6 meain staff 192 Nov 12 21:43 _layouts drwxr-xr-x 26 meain staff 832 Nov 16 17:53 _posts drwxr-xr-x 12 meain staff 384 Nov 16 17:15 _site -rw-r–r– 1 meain staff 533 Aug 10 17:59 about.md drwxr-xr-x 6 meain staff 192 Mar 20 2018 assets -rwxr-xr-x 1 meain staff 2166 Nov 13 10:55 favicon.png -rw-r–r– 1 meain staff 1291 May 23 2017 feed.xml -rw-r–r– 1 meain staff 863 Aug 27 2017 index.html -rwxr-xr-x 1 meain staff 13 Oct 11 20:16 run-dev

btw a Nu shell output of ls will look something like this

—+—————+——+–––––+———+————+————

| name | type | readonly | size | accessed | modified

For the below commands, assume I am doing ls -l | tail -n+2 when I type ls

Selecting specific columns # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#selecting-specific-columns) Lets say you just wanna see only the name and the size, what you can do is

ls | awk ‘{print $9,$5}’ # bash ls | pick name size # nu What this does is that it selects the 9th and 5th column from the above output separated by spaces. The output of this will look kinda like below.

LICENSE.md 1077 README.md 139 _config.yml 1605 _includes 416 _layouts 192 _posts 832 _site 384 about.md 533 assets 192 favicon.png 2166 feed.xml 1291 index.html 863 run-dev 13

Now, if you need this in column format you can just pipe the output into column

ls | awk ‘{print $9,$5}’ | column -t ls | pick name size # nu and you output will become

LICENSE.md 1077 README.md 139 _config.yml 1605 _includes 416 _layouts 192 _posts 832 _site 384 about.md 533 assets 192 favicon.png 2166 feed.xml 1291 index.html 863 run-dev 13

caveat: if there are spaces in filenames, this will not work

Sorting the data # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#sorting-the-data) OK, lets say we need to sort the files by size or by name. What would you do? For both the use cases you can make use of the sort command.

If you need to sort by size, you just give the following command.

• -n: say that it is a numerical sort

• -k: specify which column to use for sort

ls | sort -nk5 # bash ls | sort-by size # nu This will give you something like the following

-rwxr-xr-x 1 meain staff 13 Oct 11 20:16 run-dev -rw-r–r– 1 meain staff 139 May 27 2017 README.md drwxr-xr-x 6 meain staff 192 Mar 20 2018 assets drwxr-xr-x 6 meain staff 192 Nov 12 21:43 _layouts drwxr-xr-x 12 meain staff 384 Nov 16 17:15 _site drwxr-xr-x 13 meain staff 416 Nov 13 10:56 _includes -rw-r–r– 1 meain staff 533 Aug 10 17:59 about.md drwxr-xr-x 26 meain staff 832 Nov 16 17:59 _posts -rw-r–r– 1 meain staff 863 Aug 27 2017 index.html -rw-r–r– 1 meain staff 1077 Aug 10 17:59 LICENSE.md -rw-r–r– 1 meain staff 1291 May 23 2017 feed.xml -rw-r–r– 1 meain staff 1605 Aug 10 18:47 _config.yml -rwxr-xr-x 1 meain staff 2166 Nov 13 10:55 favicon.png

You could the probably just pick only the name and size from it. The command for that would be same as above.

ls | sort -nk5 | awk ‘{print $9,$5}’ | column -t # bash ls | sort-by size | pick name size # nu and this would give you

run-dev 13 README.md 139 assets 192 _layouts 192 _site 384 _includes 416 about.md 533 _posts 832 index.html 863 LICENSE.md 1077 feed.xml 1291 _config.yml 1605 favicon.png 2166

Pretty neat, huh?

Instead if you where to sort by name, you would have one less parameter to sort. You will not have the -n flag as it is not a numerical column.

So, if you need to do that, the command would be:

ls | sort -k9 # bash ls | sort-by name # nu first and skip # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#first-and-skip) Well, this is pretty much head and tail.

• To take the first n items, you would to head -n

• To take the last n items, you would do tail -n

• To skip the first n items, you would do tail -n+<number+1>

As simple as that. Lets say you wanna sort by size and take the first 5 and skip the first two after that. It would look something like this.

If you really wanna go there you can always use sed 11q instead of head. One for the memes.

Picking the nth item # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#picking-the-nth-item) Nu shell has this command called nth which lets you pick the nth line in a list. You can do something similar by using awk.

Lets say you wanna pick the 5th element

ls | awk ‘5 == NR’ # bash ls | nth 5 but awk is actually even more powerful, lets say you wanna pick every even item, you can do

ls | awk ‘0 == NR % 2’ # bash

NR in awk is a special variable, it give you the current line number. You can do any math operation on it. In here we just take the modulus of it with 2 and prints if it is 0.

Check if column value greater than a specific value # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#check-if-column-value-greater-than-a-specific-value) Nu shell provides this command called where which you can use to compare values like an SQL where clause. You can actually do that with awk.

ls | awk ‘$5 > 1000’ # bash ls | where size > 1000 # nu Ideally the command would look more like

ls | awk ‘$5 > 1000 {print $0}’ but the {print $0} part will be put there if we do not have anything in there.

Sum up values in a column # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#sum-up-values-in-a-column) Again, you can just use awk for this. Maybe this is a blog about awk than bash?

So, lets say you wanna view how much cpu each process takes and sum it up. If you run the ps command it will give you this data. For our use let us just get the command name and the cpu usage.

Running

ps axhc -o command,%cpu -r will give you something like the following

COMMAND %CPU firefox 67.3 -zsh 13.0 WindowServer 6.9 hidd 1.4 plugin-container 1.1 tmux 0.9 karabiner_grabbe 0.9 …

Now, if we needed to sum up the values of cpu, we can use the following command

ps axhc -o command,%cpu -r | awk ‘{s+=$2} END {print s}’ OK, let me explain. We just create a variable s and add all the values of $2 which is the second column item (cpu) and once we are done, we print out s. Its as simple as that.

Meta # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#meta) What if the separator is not space? # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#what-if-the-separator-is-not-space%3F) Well, in most common cases, the separator will be space. But what if it is not? You can pass in a argument to most of the commands to say what the separator is.

I am going to use a file which looks like below for demo. Just pulled from their page.

For column you can use -s. # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#for-column-you-can-use--s.) If you wanna see this in columns, you can just do

column -s’|’ -t people.txt # bash open people.txt | lines | split-column “|” # nu The output for this will be like what you would expect for it.

Octavia Butler Writer Bob Ross Painter Antonio Vivaldi Composer

For awk you have -F # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#for-awk-you-have--f) You can just pass the separator using the flag -F.

Lets say you just wanna view to the first column, you can just do.

awk -F’|’ ‘{print $1}’ people.txt The output will be, as you expect

Octavia Bob Antonio

Just as a reminder, this is not the only way to do it. Here are a few other possible ways.

cut -d’|’ -f1 people.txt sed ‘s/\ |.*//’ people.txt grep -o “^\w*\b” people.txt grep -Eo ‘¹+’ people.txt For sort you have -t # (https://blog.meain.io/2019/nu-like-table-operations-in-bash/#for-sort-you-have--t) If you wanna sort by the third column, you can run

sort -t ‘|’ -k3 people.txt

you will essentially get you

Now pass it to column for a cleaner output

sort -t ‘|’ -k3 people.txt | column -s’|’ -t

and that will give you

Antonio Vivaldi Composer Bob Ross Painter Octavia Butler Writer

And that is a wrap, I guess I convinced you that BASH is really powerful. Hopefully, I got at least a few more people to write more BASH.

^| ↩

Reference: https://blog.meain.io/2019/nu-like-table-operations-in-bash/

Write a comment

No comments yet.