Find. The art of searching in the Linux filesystem

There's no doubt that searching files is a primary need on every system, whether is a website ,a storage, a database or the entire internet. Linux OS provides several very useful tools for the user to perform detailed and effective search operation against the entire filesystem and beyond.

Linux Find

"find" command is the main Linux filesystem search tool. The structure of the command is the following: 

#find |directory| |-switch1| |-switch2|.... |-switchN|

So the first part is just the command find, on the second part is declared the directory where the search will be performed .Then on the third part consists of one ore more switches according to the search terms such as filename, filetype, creation time etc. On the last command switch the N switch, there's an option to execute commands over the search results.

A few examples

#find /home/user1/bucket -name tool 

This is a simple search over the folder bucket under user1 home directory, searching for the item (file or folder) named "tool".

#find /home/user1/bucket -name tool -type f

Now the system will search only for files named tool. If we change the "type f" to "type d" then it will search for directories named tool. 

#find /home/user1/bucket -name *.conf -type f

Of course there's also a wildcard option, which in this case we search for all the .conf files in the bucket directory.

#find /home/user1/bucket -name *.conf -type f -mtime 7

Here again the system will search only for .conf files but only .conf files that have been modified 7 days ago.

#find /home/user1/bucket -name *.conf -type f -mtime 7  -exec cp {} /home/user2/ \;

At last we can take the above search and perform a copy over the detected .conf files modified 7 days ago, with destination the home directory of user2 user.

Locate

Mlocate is an ultra fast utility which can help the user to easily find any file on the system without even having a clue in which directory it may reside. It achieves that by indexing all files with corresponding paths on a single database. Although it lacks the capabilities of find command, it surpasses it in speed and simplicity. So let's start by installing this utility.

#yum install mlocate for CentOS , RHEL, Amazon Linux

#apt install mlocate for Ubuntu and Debian based

#dnf install mlocate for Fedora

After the installation we need to force a database indexing by giving:

#updatedb

The update of the database has to be manually always. The only way that is performed automatically is only during a system reboot.

Now lets say that I need to change my DNS resolver. I just remember that the name of the conf file is resolv.conf but I don't have a clue where this file is so I can just give:

#locate resolv.conf

and within fractions of seconds I got the result which of course is: /etc/resolv.conf

Now let's assume that I've installed apache, but this is my first time with this program and I don't even know where are the related directories have been installed. After updating the database with the updatedb command I can give 

#locate -i apache 

And I got all the files and directories , with full path, containing the word :apache". Switch -i is for ignoring case. 

Grep

Grep is a very powerful command to search inside files. It is very using for log reading, scripting even manipulating the contents of a file. The basic syntax is the following:

#grep nameserver resolv.conf

This command will search inside resolv.conf file for the word nameserver and it will return the entire line of each finding. eg the output will be something like:

nameserver 1.1.1.1

nameserver 8.8.8.8

#grep -v nameserver resolv.conf  

Will return the exact opposite of the match, so it will hide the above result and display all the rest of the file.

#grep -o nameserver resolv.conf 

Will return only the matching words so in our case the output result will be:

nameserver 

nameserver 

#grep -A 1 1.1.1.1 resolv.conf 

Will return one line after the specified match  "nameserver  8.8.8.8" and

#grep -B 8.8.8.8 resolv.conf

Will return one line before the match, thus "nameserver 1.1.1.1"

Last, but not least a very powerful option is :

#grep -r nameserver /etc

This command will search recursively the entire /etc/ directory for the string "nameserver" and it will return the full path of each file along with the line containing that string eg in the resolv.conf case the output will be: 

/etc/resolv.conf:nameserver 1.1.1.1

/etc/resolv.conf:nameserver 8.8.8.8 

IPtables , The Legendary Firewall

A Brief History

Iptables based on netfilter framework has become the default for firewall software in Linux for nearly two decades. Netfilter/iptables framework is a kernel module supported since the 2.3 version, developed by Rusty Russel back on 1999. Here you can check his personal blog: https://rusty.ozlabs.org


A Strong Security Solution

Iptables is a very reliable and secure software, and it is remarkable that is not used only as a local machine firewall. Linux OS, together with iptables, installed on a machine can be used as a hardware router/firewall solution also. In addition to that there are even some open source software appliances used as a router/firewall based on Linux and iptables.


Netfilter Architecture

The Netfilter architecture is divided in the three following layers:Tables-Chains-Rules, see picture below

 

At the lower level we have the tables, which represent the type of packet processing that is happening through the firewall. The basic tables that are frequently used are the following:

- Filter: Table for packet filtering.
- NAT: Table for NAT rules.
- Mangle: Table for mangling packets.

At the next layer there are the Chains, which are simply lists of rules associated with each particular table. And finally, at the top there are the actual firewall rules controlling the access to the system.


Iptables in use


So let's try to see if we have any iptables rules loaded in our system:
give

#iptables -L

and you'll get something like this:

Chain INPUT (policy ACCEPT)
target                                              prot opt source               destination        
KUBE-FIREWALL                           all  --  anywhere             anywhere           

Chain FORWARD (policy DROP)
target                                            prot opt source               destination        
DOCKER-ISOLATION                   all  --  anywhere             anywhere           
DOCKER                                       all  --  anywhere             anywhere           
ACCEPT                                        all  --  any where             anywhere     ctstate RELATED,ESTABLISHED
ACCEPT                                        all  --  anywhere             anywhere           
ACCEPT                                        all  --  anywhere             anywhere           

Chain OUTPUT (policy ACCEPT)
target                                          prot opt source               destination        
KUBE-FIREWALL                       all  --  anywhere             anywhere           

Chain DOCKER (1 references)
target                                         prot opt source               destination        

Chain DOCKER-ISOLATION (1 references)
target                                         prot opt source               destination        
RETURN                                    all  --  anywhere             anywhere           

Chain KUBE-FIREWALL (2 references)
target                                         prot opt source               destination        
DROP                                          all  --  anywhere             anywhere         



On this particular host there are some rules generated from Docker and Kubernetes deployment.
First column is the Chain. Second column (prot) describes the protocol involved with the rule, third column (opt) is the ip options. Finally source and destination represents the source/destination IP or subnet involved with the rule.

Now lets say we need to blacklist an IP address so that our host is blocking every incoming and outgoing packet to this particular IP


#iptables -A INPUT -s <ipaddress to block> -j DROP


In that rule the firewall simply drops every incoming packet from the blacklisted ip address.  The rule is categorized under the FILTER table, and under the INPUT chain. The -s switch is used to filter the source ip address.


#iptables -A OUTPUT -d <ipaddress to block> -j DROP


In that rule the firewall drops every outgoing packet with destination IP same to the  IP address that is blacklisted. This rule is categorized under the FILTER table as well, but this time is under the OUTPUT chain. Now instead of source , we have destination ip address, thus the -d switch.


In some other occasion we may need to allow ssh connections to our host

#iptables -A INPUT  -p tcp --dport 22 -j ACCEPT


or block ping requests

#iptables -A INPUT -p icmp --icmp-type echo-request -j DROP

So you get the idea how this works.


Iptables as a router.

In that case it is possible to use a Linux box as a gateway to route LAN traffic to internet. For this operation you can utilize chains PREROUTING and POSTROUTING .


PREROUTING chain controls the incoming network packets from the LAN to the Linux box. So in practice you use PREROUTING for port forwarding in most cases
for example:


#iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to 192.168.1.10:80
 

this rule is allowing a web server to operate , by simply redirecting HTTP traffic from outside the LAN (internet) to the port 80 of the web servers IP (192.168.1.10).

POSTROUTING chain controls the outgoing network packets from the Linux box to the internet and is the chain which routes all the LAN traffic outside , so here we talking about the NAT process, so the rule goes

#iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
 

Based on that you can create your own firewall appliance just by using a single PC with a couple of network interfaces.

SSH Key Based Authentication

There is a big debate whether is better to use passwords or SSH keys to login on your Linux systems.

Well, in my opinion key based passwordless authentication is mandatory when you have to deal with network automation and mass configuration tasks, like Ansible scripting, or automated secure copy (scp). Is also easier than typing passwords all the time and more productive especially in large scale infrastructures. But when it comes to security things are more complicated.

First of all, a few words about SSH. SSH or secure shell is a network protocol which uses public-key cryptography to establish secure connections between a server and a client. It is commonly used in Linux and Unix systems of course, but also in most of the major cloud services.

All we need to implement this is to create a (public – private) key pair. I keep my private key secret in my system and I send my public key on the server, the algorithm matches the key and I can be authenticated. 

 Now let’s do some magic and make our machines login and send files through ssh without the use of a password. So I’m going to log in to my charming Linux Mint desktop, and then create that pair by giving 

# ssh-keygen –t rsa

Now we get an interactive prompt asking us to enter some info:

Generating public/private rsa key pair.

Enter the file in which to save the key (/home/user/.ssh/id_rsa):

Here we just pressing enter to accept this directory. Usually Linux systems keep the key-pair in the hidden ssh directory under the home directory of the user.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Here we can give a passphrase to encrypt our private key for extra security.

After that we get the funny randomart image on our terminal which indicates that our key-pair is ready.

Now if we navigate on our keystore directory we can find our key-pair

# ls –ltr /home/user/.ssh/

Id_rsa.pub is the public and id_rsa the private key accordingly.

As it said before we need to keep our private key secret, and all we have to do is to send the public key to the server we want to login. 

On the server side now, we navigate to the user profile we need to use for auto-login. On my Centos server is /home/remoteuser/.ssh/

Now there should be a file authorized keys, if not create it with 644 permissions:

#touch authorized_keys

#chmod 644 authorized_keys

Finally copy your public rsa key and paste it (plane text) inside this file. Now you must be able to login without password, try

# ssh remoteuser@mysever.local

 Enjoy

As an epilogue I can say SSH (key-only) based authentication is great in respect of security and can keep your servers unaffected from brute force attacks or man in the middle attacks.

But what happens if a private key is leaked or a client workstation gets compromised?It's pretty the same as losing the keys of your house.

So the choice is yours to decide according to your environment and your needs.