Linux Bash type and aliases

Something I learnt today about aliases in Linux Bash shell. I use plenty of aliases in bash. Sometimes I forget what a particular alias stands for. Apparently the which command doesn’t work for this purpose. As explained here: bash-aliases, you can certainly do alias <alias-name> to check what it does. However, this doesn’t work for any custom functions we define in bash. The command type works in all cases and is the best utility to use for this purpose.

type <alias/function> will list out the exact command in all cases.

 

Advertisements

Handy Linux tricks with find

I often find the need to delete files of a particular size. Instead of viewing files in a graphical interface, arranging by size and then picking the ones to delete, there is an easier way. This is taken from the following SE posts:

http://unix.stackexchange.com/questions/37627/delete-files-of-certain-size-range

As mentioned there, the + or - prefixes are neither documented in the man pages nor within info. So in order to find files greater than 10MB but less than 100MB, use this:

find  .  -size +10M -size -100M  -exec rm -f  -- {} \;

The syntax of why the backslash and semi-colon is required is described here: semi-colon within find. The double-dash (--) tells the getopt interpreter that rm uses to ignore anymore options, and so allow deletion of filenames that begin with a dash(-). (Why would anyone have filenames like those👺: rm with special characters)

Alternatively, you could use xargs for the deletion.

find  .  -print0  -size +10M -size -100M  |  xargs  -0 rm -f  --

The arguments “-print0” and “-0” are required to interpret files with spaces in the names correctly. (find and xargs).

If you want to list all files based on their modification times, use this:

find  .  -type f  -mtime 0

will show all files in your current directory modified (or created) within the past 24 hours. If you’d like find to count from the beginning of today (rather than 24 hours ago), use the option “-daystart” before any time options.

The same + or - prefixes can be used here to limit the search to strictly greater/less than conditions.

ps: David Wheeler has written an excellent article on the ridiculous use of certain characters in POSIX filenames: No Sharp Corners

Linux file attributes and creation time :)

I didn’t know about the Linux ‘chattr‘ command which allows you to display and set file attributes. Using this, I learnt a neat trick on how to make a file immutable, even by the root user.

To make a file immutable, do:

chattr +i <filename>

To remove the immutable attribute, do:

chattr -i <filename>

To list the current file attributes, do:

lsattr foo/bar

It’s that simple 😎

This will change only the ctime for the file, and not its mtime. (details here: ctime vs mtime). Another interesting thing, in th Raspberry Pi context, is the atime file information. Essentially, atime should reflect the last time a file has been accessed. However, this is not very useful and only serves to increase the number of writes to a partition, every time a file is accessed. This is a dampener for performance esp. in embedded systems. Hence, on the Raspberry Pi, we mount partitions with the “noatime” attribute within fstab.

In Linux, the file creation time didn’t exist until the ext4 filesystem appeared. ctime originally did mean creation time, but it always reflected the time when a file’s metadata was changed (attributes, permissions, etc). With ext4, the creation time was stored in the file’s inode, under the field ‘crtime’. In order to get a file’s creation time we need to lookup the file’s inode, and the use stat and debugfs to find the required field. debugfs is required since there still doesn’t exist any kernel API to get crtime from stat. (birth time)

ls -i <filename>
debugfs -R 'stat <inode_number>' /dev/sdNN

(replace sdNN with the actual path to the device where the file resides). Look the the ‘crtime‘ field in the output.

Bash shell tricks

The bash shell has so many nifty tricks. They can come in handy in order to save time with repeated tasks, recover data, etc.

  1. My set of customizations for bash can be found on github. With those customizations, you can make find easier, colorize diff, man and prevent clobbering when using output redirection. I especially love the colored man pages.

  2. If you want to sudo-ize the previous command, use;

sudo !!
  1. rename utility: I never knew this! It makes renaming multiple files using regex patterns so much easier!

  2. In scripts, instead of the awkward test command ‘[‘, use ‘(( … ))’ and ‘[[ … ]]’ in its place. It makes logical and string comparison so much easier. (ref: modern bash test operators)

  3. To get your current shell name, do

ps -p $$

or

echo $0

 

Some more examples can be found here: bash trick commands

There are also some intricacies like the difference between rm and unlinkrm vs unlink

Feel free to suggest more in your comments.

Note: The file .bashrc is not loaded by bash at startup, by default. So it must be included in either the file .profile or .bash_profile.

ssh tricks

I must have used ssh for the past 15 years but I never knew this trick. So if you want to ssh into a remote server and run some command on it, and exit, you don’t actually need to login into the console and type the commands. It can be done by injecting commands through a ssh connection (and piping the output back too).

So if you want to ssh into the raspberry pi and do a simple ‘ls’, use this:

ssh  pi@host -p port#  'ls'

If you use key authentication and have saved the keys, your login can be automated without the need for password (more on that later).

If you’d like to get the output and store it on the local host, use regular linux redirection:

ssh  pi@host -p port# 'ls' > test.txt

Isn’t this great 🙂

More details can be found here:

Run commands through ssh

Pipe output from local to remote server using ssh

Backup, backup and wait for it…. backup!

We must remember to backup the RPi’s SD card! First of all, SD cards were never meant to be used the way they are in the RPi. They were meant for data blocks written infrequently by cameras. USB flash drives are better suited for RPi-kind data storage than SD cards.

Secondly, the RPi’s data bus is very finicky when it comes to the input voltage. I was trying to power my Pi using a portable mobile phone charger by HooToo (hootoo). Though it’s rated output is 2A, I think I connected and disconnected it multiple times by mistake without a clean shutdown. This corrupted my sd card! But I am not talking about just data corruption – it physically corrupted it such that it cannot even be read anymore.

So the lesson learnt is – backup as much as possible!

The simplest way I know is to use a Linux computer and the ‘dd’ command. Here are some steps to follow:

Make sure there’s enough space on the local drive!

Check partition names:

sudo fdisk -l

Unmount SD-card partitions before reading or (esp.) writing!

sudo umount /media//root

Use this for backup:

sudo dd if=/dev/mmcblk0p6 of=root.img bs=32M

(The block size could also be 4M)

Use this for restore:

sudo dd if=root.img of=/dev/mmcblk0p6 bs=32M

For those who want to use windows for backup, the standard win32diskimager tool is great. But note that it clones the entire SD-card. You cannot select individual partitions.

Securing secure-shell server

Credit:random

A follow-up to my earlier post about securing access to your RPi. (fail2ban-lifesaver). Some amount of security can indeed be obtained through obscurity. One way to do this is to change the default port # for the ssh server. Since most hackers will try to attack port 22 first, it’ll take some time to reach the obscure port number set by you on your RPi. Yes port scanning will eventually find it, but hey, why not delay the attack 🙂

#Port 22
Port 16022

Read up about how to change the default ssh server port here (its really quite simple): Change default ssh port.

Also remember to change it in the fail2ban config file.

#port = ssh
port = 16022

Further security measures like public key, port knocking: Harden the ssh server.

I also used iptables to block a whole range of IP addresses, from which hackers are trying to attack my RPi. This is how you do it: Block a range of IP addresses.

sudo iptables -I INPUT -s 103.0.0.0/8 -j DROP

Bash shell scripting gotchas

I hate scripting in bash (or other shells) since the syntax is so terrible and I can never remember how to start coding in it. Today I learnt of a simple trick to avoid the pitfalls of not having spaces before and after the [ ] used in if statements.

More on this is at SO: bash brackets

Why bash why? Why don’t authors of popular shells just adopt a common syntax like say, C for everything? Why must we learn a million different syntax-es?

On those lines, don’t get me started on perl syntax either … 😦

fail2ban – lifesaver!

borrowed from Gizmodo

Hacking attempts and DDoS attacks are commonplace. In fact, its been just a week since I setup my RPi as an always-on device, with sshd service running. Today, I opened up the authentication logs and found 100s of login failures over ssh, all coming from China. I installed fail2ban which seamlessly takes care of banning clients with repeated login failures. It is easily configurable via a simple config file.

The attacks seems to be from a Linux Malware called XOR.DDoS (details here: XOR.DDoS)

These are the IP addresses seen attacking my RPi:

XOR.DDoS-IPs

(The number in the first column denotes the number of times the client has tried to connect and failed).

These are the messages in the auth.log:

XOR.DDoS-loginfail

Using ip2location, I traced them to:

XOR.DDoS-IP

Seriously! What are you trying to get from my RPi! Stop or I’ll have to send my attack cat to get you!

cat-tux

borrowed from rore@flickr

Everyone must install fail2ban (or equivalent) firewall programs for the always-on connected embedded devices like the RPi!