BlogPost2 LinkedIn Linux System Engineer

Motivation

Revise and strengthen my Linux knowledge. I have been using Linux system for years but never been formally trained for it.

It turns out that UQ offers unlimited LinkedIn learning access. Let’s make the best use of the accessibility while it last

Just a note of what I learned

Table of contents

A course on LinkedIn.

1. Introduction

2. Gathering network information

3. Ensuring connectivity with ping

4. Querying DNS servers with dig

5. Changing hostname, IP address, and more

6. Configuring networking

7. Connecting to SSH

8. Configuring SSH clients

9. Authenticating and restricting access with SSH

10. Optimizing SSH for speed

11. Apendix

Introduction

Copied from the course intro:

Study to pass the RHCE—the benchmark certification for Red Hat Enterprise Linux. Watch this course to prepare for the networking and Secure Shell (SSH) topics covered by the exam. Instructor Grant McWilliams provides an overview of Linux networking and key commands and tools for gathering information about your network, including Network Manager’s command-line tools nmcli and nmtui. Grant also teaches the iproute2 commands such as ip and nstat, and cross-references them with legacy tools. Learn how to use ip, nmcli, nmtui, netstat, nstat, ss, arp, and dig to get statistics and analyze data, and change live and saved network configuration settings. Grant shows how to create a custom ifcfg-ethx file for manual network configuration and uses systemd to configure the hostname. He then shows how to connect to and configure Secure Shell (SSH), wrapping with a look at SSH security and SSH tips and tricks that will help you optimize SSH and execute commands remotely.

Prerequisites:

1. Centos Linux installed on VMware. Create multiple users and create multiple VMbox.

2. Configure network within VM, create network to link multiple clone machines through full clone (linked clone also works and saves some memory) -> create at least two clones.

Note to take: Centos uses yum Ubuntu uses apt

1. wire -> wire setting to connect the machines. after configuring the machine check with ping. and ping the other machines says host1 and host2

2. update centos kernel: sudo yum -y update

3. install software DevelopmentTools : sudo yum group install -y --setopt=group_command=objects "DevelopmentTools"

4. install kernel source: sudo yum install -y kernel-devel

The theory,

Network naming: eth0, eth1, wlan0, wlan1 -> the most friendly way

Systemd naming schemes

• Onboard devices are named using firmware or BIOS info
• PCI express hotplug devices using firmware or BIOS info
• Devices are named by physical location
• Use MAC address
• Traditional unpredictable naming (old good way)

Systemd network types (more in depth Systemd)

• en = ethernet
• wl = wireless
• ww = wwan

Systemd network location

• o = onboard
• s = PCI hotplug
• x = MAC address
• P = PCI domain
• p = PCI bus number

i.e. P1en5s0 if the device was an Ethernet device in PCI domain one or enx08002772b666: ethernet device, named by mac addr ens1f4: ethernet device, PCI hotplug, function 4

Note: iptable, firewalls or kickstart call network interface by names files -> use grep -r eth0/etc/* to iterate through the list of devices

Network manager configuration control these

• nw aliases
• IP addresses
• static routes
• DNA info
• VPN connections
• connection param

The basics that you know but rarely use

1. get kernel’s name uname: print out kernel of your machine -a print out all info, v version, r release date

2. check for free disk space:
   df -ah : a for all infor, h for human readable format. (df: disk free)

3. manage services on a system:

service udev status: check the status of the service named udev

systemctl status udev: check the stt of the service

1. check the size of a directory content on disk du -sh code/: disk usage of a directory

2. check for open port, check listening network socket, service listening netstat network status

3. CPU usage for a processor ps aux | grep nginx: processor a: show processes for all users, u display the process’s user/owner, x also show processes not attached to a terminal top or htop can also do the trick, may need to install through apt install

4. mount new hard drive to linux ls /mnt check this directory mount dev/sda2 /mnt : mount sda2 to this location mnt check this file for a comprehensive list: cat /etc/fstab

5. nslookup - query Internet name servers interactively, DNS query

6. how to look up something, don’t remember or know man <command> look up the usage/manual of a command

Gathering network information

Previously, Lunux use net tools commands including ifconfig netstat route . But they are going to be deprecate soon

1. Get network basic infor, subnet mask, and default gateways

   should start to migrate to ip and route

   if you are too lazy to change (you shouldn’t Minh), install net-tools to have access to legacy commands and replace the default. sudo yum -y install net-tools

   Gather network devices

   ifconfig -> output the list of network devices

   ifconfig enp0s3 -> get more detail about enp0s3 device, like netmask

   alternatively (since ipconfig is deprecated),

   use ip addr show -> spits out the list of devices as well but looks scary

   use ip addr show enp0s3 -> work similar legacy ifconfig enp0s3

   Gather network default gateways

   route -> return tables with column flags

   route -n -> return get the gateway infor in numeric numbers

   alternatively,

   ip route -> work the same as route

2. Get system hostname

   check the host name on startup from static hostname: /etc/hostname

   Transient hostname: default stored in /etc/hostname but and manually or dynamically by DHCP

   Pretty hostname: free-form utf8 string of text that is presented to the user, google. Rules for hostname: (1) can be up to 64 char in length (2) 7-bit ASCII lowercase char, (3) no space. Limited to the combination of a-z; A-Z ;0-9; -; _; .

   hostname -> legacy

   hostnamectl -> latest changes

3. Get ntw infor from GUI -> bunch of step to get the above information from GUIs, look more friendly with a few clicks

Quiz: The hostname is managed by systemd so even though you can edit /etc/hostname manually you still have to notify systemd to update it. -> True In order to configure static IP addresses on Enterprise Linux 7 a user has to be root. -> False

Ensuring connectivity with ping

1. To ping from one host to the other: get the ip address show and value of inet to ping

2. netstat get the nw statistic (from legacy).

   Alternatively use: nstat -asz : param -a for absolute counter, -s : no history and -z for not include non-zero counter.

   nstat -a : look at all the history, l is for list

3. show listening and non-listening socket: netstat -a or ss -a

   display open nw port types: netstat -l or ss -l -> search for service listening on a port. can use grep to search over it.

   netstat -lp or ss -lp : p for process names

   netstat -t -a or ss -t -a: t for tcp socket type, replace t for u for UDP socket and for w for WA sockets.

   netstat -i or ip -s link for interface statistic

   netstat -r or ip route for routing information

Querying DNS servers with dig

1. Address resolution protocol (or ARP): is a protocol responsible for resolving IP addresses to MAC addresses

   arp (legacy command only available in net-tools ): show MAC, IP addresses and network card for all host that cached in your computer

   ip -s neighbor list as an alternative -s for status

2. Query DNS servers, (you know dnslookup but it’s not so powerfull)

   dig redhat.com: shows the information on redhat.com

   dig redhat.com +noall +answer: to filter the response with answer and noall criteria

   dig reahat.com +short: get the ip address associates with the domain name

   dig -x <ip address> +short: get the domain

   dig -f dnsname.txt +noall +answer: to get the input from files

Changing hostname, IP address, and more

1. In order Linux version: host name was stored at /etc/sysconfig

Configuring networking

Connecting to SSH

ssh: secure shell, provide the secured login to the server, communication is encrypted, even encrypts insecure protocols. Support remote file copy.

Installation in Centos: sudo yum install -y opensssh-clients or sudo yum install -y opensssh-server

If openssh is installed in the server, use systemctl to enable the ssh.services

sudo systemctl enable sshd.service then sudo systemctl start sshd.service

To setup the host machine to access through ssh: get the ip and put in the ssh ip.

Configuring SSH clients

Default ssh works just fine but you can always customise the configureation through ~/etc/ssh/ssh_config

i.e. ForwardX11Trusted can grant access to linux GUI

Client config

There are multiple ways to config the ssh client configuration.

1. Command-line option

2. Per user configuration at `~/.ssh/config`

3. System-wide configuration at `/etc/ssh/ssh_config`

-> System wise configuration

sudo less -Nq /etc/ssh/ssh_config : -N for line number, -q turn off error noti ->

Server config

Notice the difference sudo less -Nq /etc/ssh/sshd_config : -N for line number, -q turn off error noti

Config both server and client

Connect to the server for the first time (or generate ssh_key) will generate the folder ~/.ssh/config.

Manually create /.ssh/config folder through vim

Host host2
    User user1
    HostName 192.168.0.21

Host host3
    User user1
    HostName 192.168.0.21
    Port 1022
    IdentifyFile ~/.ssh/rhhost2.key

Host *
    User user1

-> the above config will save time, instead of typing ssh username@ip.host.addr

chmod 600 ~/.ssh/config then ssh host2

Authenticating and restricting access with SSH

Passwordless login with PEM

ssh-keygen -> this will create a pair of id_rsa for private key (id_rsa) and public key (id_rsa.pub) login.

ssh-copy-id user1@192.168.0.21 -> two things happen (1) copy the public key to the remote server 192.168.0.21 store it under ~/.ssh/authorized_keys file, (2) the fingerprint of the remote server is stored in local_host users under ~/.ssh/known_hosts

ssh-add -> to make sure the ssh private key is added to local directory for login.

-> all of these above is to setup login without password

Restricting root logins with ssh

Don’t let root user to login through root. Strong password is good but no root login is better

sudo vi /etc/ssh/sshd_config then search for PermitRootLogin yes and then set it to no. (should uncomment it to enable it as well).

use sudo systemctl restart sshd.service after modify the

Restricting users and network to login

Limit some user login or login through some network

sudo vi /etc/ssh/sshd_config then:

1. search `ListenAddress` modify the ip address there to limit the network can access. 

2. search bottom add line `AllowUsers userName1 userName2` -> to allow only userName1 and userName2 to login. 

    2.1 Specify user login through port: `AllowUsers userName1@192.168.0.0/24` -> user1 login through port 24

Optimizing SSH for speed + ssh tips/trick

Copy file from local to remote server or vice versa a is rather difficult.

tip 1: use scp -c arcfour -r remote.ip.addr:/etc /tmp -> RC4 cipher for better speed instead of AES, but poor encryption. (-c for cipher)

tip 2: use -p : use -p permission reserved backup data.

tip 3: use rsync to copy the whole folder. more complicated rsync -av -e "ssh -p 1000" /etc 192.168.1.247:/tmp. There are heap of param that you can test.

tip 4: use dd (duplicating a disk through ssh): dd if=/dev/sdb | ssh 192.168.1.247 "dd of /def/sdb"

tip 5: use sftp file transfer protocol

tip 6: connect to remote host and execute a shot command (wow that’s really a stupid trick). ssh user1@192.168.0.1 "echo Who" > host2-command.txt . not all command can run this way as it doesn’t execute in sudo mode.

tip 7: ssh -o PrefferedAuthentications=pubkey user1@hostname.com

Access control list Access control list (ACL) provides flexible permission mechanism for file systems: root vs users.

getfacl : get file access control list permission.

sudo setfacl -m g:xxx:rx [folder] : set file acl permission to a group -m to modified, g:xxx:[with the desire right] where g stands for group, xxx the group name, rx-> read and execute, .

sudo setfacl -m u:xxx:rx [folder]: set file acl permission to an user.

setfacl vs chmod -> overlaped a lot but have some differences.

1. `chmod` can only set permissions for a single user, a single group, and everyone 

2. `setfacl` can set permissions for multiple individual users, multiple groups, and everyone 

3. `setfacl` and `getfacl` can change the whole directory and subsequence files/directories within the same dir. chmod require recursive mode to do so. 

Proc filesystem Proc file system is a special type of file systems that presents information about processes and other system information in a hierachical file-like structure ( ls /proc list out the processes that are running with PID).

fun fact init always run at PID=1

process group : a collection of processes. Use getpgrp to return the process group ID

Systemd

Definition: Systemd is an init system and system manager that has widely become the new standard for Linux distributions.

To control the services in the systemd, use systemctl [stat|enable|stop|] : enable set the service up and run during boot time

init is system (processes, services, ) that loads during boot time using by systemd.

System also control users, user groups/ passwd

if you are root, /etc/group -> group exists in your machines, /etc/shadow enable or disable login

useradd and userdel to add or delete the users

Apendix

kill to kill a single process by id or by name

killall [-signal name using number] [process name] to kill all the processes with name

lsblk displays the list of block devices on your system. Logical Volume Management (LVM) consists of 3 layers: Physical Volume, Volume Group layer and Logical Volume. Next layer is buit on top of the other -> PV is the list of hard drive mounted -> VG is the grouped volumes using PV -> LV

LVM rain check `sudo pvscan ` short of use `df` for the volume but it will return the size of the volume -> to find and scan physical volumes. `sudo pvcreate [volumne name]`. 

LVM check volume groups -> scan what volume group exist `sudo vgscan ` to scan the list of volume group. `sudo vgcreate [name of volume group] [partition assigned to volume group]` 

`sudo lvscan` and `sudo lvcreate` work similar but designated for LV layer. `sudo lvcreate --name $NAME --size XXG VG` -> `--name` refers to name of logical volume; `--size` refers to the size allocate to the volume in `XXG`  

`sudo dsktype`

strace: trace system calls and signals.

rsync and scp are similar in term of purpose. But rsync copies file faster using a special delta transfer algorithm. i.e. rsync A host:B

ssh: (use that like everyday but don’t bother to learn the theory)

ssh-copy-id: use locally available keys to authorise logins on a remote machine, basicaly to copy ssh-public-key to the server