General Linux Commands

SUMMARY

General Linux Commands

ISSUE

General Commands

ls

lists files, directories, etc. within a directory.

-l long listing showing permissions, size, etc. (required for all the rest of these flags)

-h human-readable values for size

-S sort by size, largest to smallest

-t sort by time modified, newest to oldest

-r sort by reverse of other flag (i.e. ls -ltr gives files sorted from oldest to newest)

-A show all files, including hidden files (will not show . and .. as -a does)

-R recursive listing, shows all files in all directories and subdirectories of the current path

-i shows inode numbers (inodes are structures that store information about files, directories, etc.)

cd

changes directory

cd .. changes to parent directory

cd <relative path like backups> goes to relative path within directory

cd <absolute path like /backups> goes to absolute path

cd without any flags goes to the logged-in user's home directory (also represented by ~)

cd - goes to previous directory (directory before the last change of directory)

mkdir

create directory

mkdir /tmp/1 would create a directory called 1 in /tmp  

-p creates parent directories as needed

mkdir /tmp/1/2/3/4 would create all of the needed parent directories if they did not exist already  

-v verbose

rmdir

remove directory

rmdir /tmp/1 would remove that directory (if it is empty).  

-v verbose

mv

moves and/or renames files/directories

-f forces the move--no prompt before overwriting

-v verbose; show what is being done

cp

copies files, directories, etc.

-p preserve mode, ownership, timestamps when copying

-R or -r copies directories recursively

-v verbose

pwd

print working directory (current directory)

rm

removes files, can remove directories as well

-f force removal--do not prompt

-r recursive removal; allows removal of files and directories. BE CAREFUL WITH THIS!! KNOW WHERE YOU ARE!!

-v verbose

man

shows manual pages for a given program or topic

running man tar will show the manual pages for the archiving program tar

cat

concatenates multiple files, prints files to screen

-A shows all non-printing characters like tabs, end-of-line characters, etc.

-n numbers all lines

head

show the first part of a file; by default the first 10 lines are shown

-n <#> will give the first # lines of a file

-<#> will also give the first # lines of a file

i.e., head -n 15 a.txt and head -15 a.txt give the first 15 lines of a.txt

tail

show the last part of a file; by default the last 10 lines are shown

-n <#> will give the last # lines of a file

-<#> will also give the last # lines of a file

-f will follow a log file as new data is added to it, outputting the data to screen

i.e., tail -n 15 a.txt and tail -15 a.txt give the last 15 lines of a.txt

less

less is a pager program that will let you move back and forth within a file

i.e., less a.txt will open up a.txt and allow you to page up and down within it.

Note: typing gg goes to the beginning of the file, G goes to the end, q quits

date

shows the current date and time

to convert from a Unix epoch timestamp to a human-readable format, type date -d @<UNIX epoch timestamp>

uptime

shows how long the system has been up, and the system load average over the last 1 minute, 5 minutes, and 15 minutes

top

gives you a running listing of the current processes running, sorted by CPU usage

ps

similar to top, except that the listing is static.

ps aux is the way I most often run ps.

ps -leaf is another common way to run ps.

ps aux | grep <processname> will show you information for the given processname.

clear

clears the screen

df

shows disk free space, disk used space, etc.

-hT will give you human-readable values and show the filesystem type.

-BG will give you block sizes in gigabytes.

du

will show disk usage for files, directories, etc.

-s gives high-level summary (1 directory deep)

-h gives human-readable information.

-BG gives block-size in gigabytes

-c gives a total at the bottom.

du -shc * is very helpful.

free

shows free memory

-m shows memory in megabytes

echo

prints to screen

-e allows for escape characters (like \n for newline)

to print the contents of a variable called BASH, do

echo $BASH  

to append words to a file called a.txt

echo end of file is here >> a.txt  

bc

calculator program best used with echo. by default only gives integers. to give numbers after decimal use scale

divide 86763 by 843 with 3 decimal points in the result

echo scale=3; 86763/843 | bc  102.921  

divide 8000 by 3600 and just leave whole number

echo 8000/3600 | bc  2  

to convert hex FF3 to decimal

echo ibase=16; FF3 | bc  4083  

to convert 1027 to hex

echo obase=16; 1027 | bc  403  

Bash Basics

Bash is the default command line shell for Linux. As you will be logged in via Putty to many backup appliances, you will become very familiar with bash.

• To redirect the output of a command to a file, use the > sign.

b.sh > a.txt will create a.txt and put the standard output of the script b.sh into the file a.txt

• >> is similar to > but instead appends to a given file.

b.sh >> a.txt will add the output of b.sh to a.txt

• | allows you to run additional commands on the output stream of the previous command. You can use as many pipes as desired.

ps aux | grep ssh will run a process listing and then look for the lines containing ssh with the grep utility.

• using backticks ` will do command-line substitution--that is, it will run the command within the backticks

i.e., one helpful command to run will give you a listing of files recursively in a given directory, and sort by time.

ls -lhtr `find . -type f`  

Using Bash History

the history command will show command history

!! runs previous command

!-5 runs the fifth to last command

!ls runs previous command starting with ls--BE CAREFUL WITH THIS

^abc^def substitute def for abc in the previous line

!123 runs history item 123

Ctrl-R allows you to search your history from most recent command to oldest

hit Ctrl-R, type vim, and if you do not see the right vim command, type Ctrl-R again and it will show you the command for vim prior to that

Ctrl-K kill (erase) rest of line

Ctrl-U erase part of line before cursor

Ctrl-X, Ctrl-U to undo your previous command line edit

Ctrl-A to go to beginning of line

Ctrl-E to go to end of line

Alt-F to go forward a word

Alt-B to go back a word

Alt-shift-8 (Alt-asterisk) gives all possible tab completion matches

Ctrl-Alt-E expands ~ (to the path for the home directory) and asterisks (shell globbing) and command line substitution

vim basics

The program vim is a fantastic command-line editor. You can run vim <filename> to open the editor. The editor is opened in command mode, and you can change to insert mode through various methods--most often by hitting the letter i. To return to command mode, hit the ESC key. I strongly recommend using the built-in program vimtutor as this will allow you to learn more about vim; just type vimtutor. While in command mode, you can delete, run special commands to look for lines containing certain information, etc. All of the following is for when you are in command mode.

• :w saves the file
• :q! quits, discarding changes
• :q quits
• :wq saves the file and quits
• :e! reopens the current file, discarding changes
• i goes to input mode at current character
• A goes to input mode at the end of the current line
• x deletes the current character
• dd deletes the current line
• u undoes the last change; this can be run multiple times
• Ctrl-R is for redo
• 0 goes to the beginning of the line
• $ goes to the end of the line
• gg goes to the beginning of the document
• G goes to the end of the document
• /searchpattern searches forward in the document for the searchpattern
• ?searchpattern searches backward in the document for the searchpattern
• :n goes to the next file if multiple files have been opened.
• :N or :prev goes to the previous file in multiple files
• :ar shows the list of files that are being edited (the vim command-line arguments)
• :set list shows nonprinting characters
• :set ic makes searches case-insensitive.
• :set hls highlights searches (default behavior most of the time)
• :set noh stops highlighting the given search pattern
• :set nu shows line numbers
• :set ff=dos converts EOL format to dos
• :set ff=unix converts EOL format to unix
• :g/regexp/d deletes all lines containing regexp
• :v/regexp/d deletes all lines except those containing regexp
• :%s/regexp1/regexp2/g substitutes regexp2 for regexp1 every occurrence in the document.
• :history shows vim history

You can also search through history by hitting / or ? or : and using the up and down arrows.

screen

screen is another great Linux utility. This will allow you to work through multiple windows on a given system. Ctrl-A is the primary meta key that is used for screen.

Ctrl-A, c to create new screen  Ctrl-A, :caption always--this gives a caption at the bottom of the screen  Ctrl-A, Ctrl-A to flip between windows    Ctrl-A, <window #> to go to that window  Ctrl-A, n goes to next window  Ctrl-A, p goes to previous window  Ctrl-A, ' to switch to a given window  Ctrl-A,  to show all windows    Ctrl-A, d to detach screen  Ctrl-A, K to kill current window    Ctrl-A, :number <#> to change order of screen  

exit will exit a screen and show screen is terminating when you have exited the last screen

screen -ls lists screens. if there are multiple, specify the PID for the one you would like to connect to  screen -x will allow you to join a current screen session  screen -d -r will allow you to detach and reattach a screen  

Parsing through Text Files

grep

search files for patterns (regular expressions)

-i ignore case

-v find all lines except those containing this pattern

-l list files where matches are found, not individual lines from the files

-A<#> for context list # of lines after match was found

-B<#> for context list # of lines before match was found

-n show line number of file where match was found

-r recursive--find files in initial directory and all subdirectories

grep <pattern> <filename>  

egrep allows for extended regular expressions and easy searching for multiple patterns

ps aux | egrep 'tasker|devmonitor'  root      1035  0.0  0.0 115544  9680 ?        SNs  Sep29   2:17 /usr/bp/bin/tasker  root     18802  0.0  0.0 108800  5216 ?        SNs  Sep29  13:34 /usr/bp/bin/devmonitor  root     30530  0.0  0.0  61188   832 pts/4    S+   20:09   0:00 egrep tasker|devmonitor  

sort

sorts lines, usually alphabetically by default using the first field in each line

-u show unique lines only

-n numeric sort

-r reverse order

-k<#> sort by this field number

cat a.txt | sort -nr -k2 would sort the lines in a.txt numerically in reverse using field 2  

sed

stream editor, can be used to replace strings and manipulate files and output streams

-i inline, perform this on a file rather than throwing the output to the screen (standard out)

to substitute the number 7 for 4 throughout an entire file you would do the following (g is for global rather than the first occurrence per line)

sed -i a.txt 's/4/7/g'  

Network Commands

nmap <IP address or host name> will show ports open/closed at a given IP or host name

Processes and Files

kill

kills a process by process number.

i.e., kill 5900 kills process with PID of 5900.

the default kill signal is -15 (SIGTERM, which allows process to cleanup), but if all else fails when trying to kill a process, kill -9 (SIGKILL) is a rude way to kill a process.

killall

killall is similar to kill but allows you to use a process name, and it will kill multiple processes with that name

file

will tell you what kind of data is contained in a file (tar archive, etc.)

just run file <filename>

chmod

change permissions on a file

+x will make a file executable.

chown

change ownership of a file/directory, can point to user and/or group

chown root /backups/a.txt would change ownership of a.txt to root.

chown -R root:root /home would change ownership of all files in /home and subdirectories to the root user and root group.

ln

creates hard links for files or soft (symbolic) links for directories or files

typical format ia for symbolic links is

ln -s <target path> <path for link to be created>  

example--you could remove logs.dir and create a symbolic link in its place to a new directory that you had created as /backups/logs.dir

ln -s /backups/logs.dir /usr/bp/logs.dir  

mount

mount drives or network shares, etc.

typical format is

mount <target> <mount point>  

i.e.

mount /dev/mapper/vgbackup-lvbackup /backups  

If a drive is already listed in /etc/fstab, you can often just use the mount point to mount it.

i.e.

mount /backups  

umount

unmount drives, network shares, etc.

umount /backups  

Note: If you get an error about files being in use, you may need to cd out of that directory. Also you can use lsof to see what files are open.

lsof

list open files

common usage:

lsof | grep <mount point>  

i.e.

lsof | grep /backups  

find

utility for finding files

typical usage is

find <what directory to start from> -type <d for directory, f for file, etc.>  

i.e. to find files with a name containing 123 or 456, do the following:

find /backups -type f | egrep '123|456'  

tar

used to create or extract from archives

to create a tarball (tar of gzipped files) of /usr/bp/logs.dir at /tmp/logs.tgz

tar -czvf /tmp/logs.tgz /usr/bp/logs.dir  

to extract from a log file (normally extracts as relative paths to the current directory, and will overwrite. BE CAREFUL!)

tar -xzvf /tmp/logs.tgz  

view files within an archive

tar -tvf /tmp/logs.tgz  

rpm

redhat package manager for installing, upgrading packages, etc. Be very careful with rpm and check with a senior engineer before doing anything!

show all packages on system

rpm -qa  

show all unitrends packages in order

rpm -qa unitrends* | sort  or  rpm -qa | grep unitrends  

update packages

rpm -Uvh *.rpm  

yum

easier way to update packages, especially when dependencies are needed etc.

yum list updates shows updates available

yum update updates all packages with updates available

yum update unitrends* updates all unitrends packages

yum localupdate updates packages from a local directory. Sometimes this is a workaround if packages have to be transferred via an unusual route (if WAN access is not available, etc.)

File Transfer

wget

download files using ftp

wget ftp://ftp.unitrends.com/bp/latest_build/airbag.tar  

show current IP address

wget -qO - icanhazip.com  

get multiple files using dash (reads from standard input)

wget - ftp://ftp.unitrends.com/bp/latest_build/Windows/PC61_Universal.exe ftp://ftp.unitrends.com/bp/latest_build/Windows/md5sums  

scp

secure copy protocol, use to securely transfer files using ssh

-P port number

-C compress

to transfer abc.txt from a customer DPU to your home directory on the tunnel server. use your username rather than jsnipes

scp -P 222 -C abc.txt jsnipes@tunnel.unitrends.com:~/  

to grab output.txt from a remote system and transfer it to current directory

scp -C root@uvault1a:/tmp/output.txt .  

NOTES

Commands only