Linux: Working with sed

By -


Working with sed (a stream editor)


  At some point I had to modify the configuration of multiple nginx configuration files, replacing on all of them a specific string with another one, commenting some strings, deleting some other strings, on one of my CentOS machines.
  If I would go to edit manually file by file, it would take a few hours, at least.
  So, I choose to use sed and to edit all files at the same time and in order to explain, I creates 100 files containing the same string, I'm using "tee" command:
johnyc20@centos:~$ echo "fastcgi_pass 127.0.0.1:9501;"  | tee site-name-{1..100}.conf
  Now I have 100 files containg the "fastcgi_pass 127.0.0.1:9501;":
johnyc20@centos:~$ cat site-name-*
fastcgi_pass          127.0.0.1:9501;
  Now I will explain:
  1. How to replace content
  2. How to comment lines
  3. How to insert lines 
  4. How to delete lines
  5. Short summary of sed options

How to replace content

The first thing, I am replacing the '127.0.0.1' string with '192.168.200.10':
johnyc20@centos:~$ sed -i 's/127.0.0.1/192.168.200.10/g' *.conf 
Now all my files now contain "fastcgi_pass 192.168.200.10:9501;" line:
johnyc20@centos:~$ cat site-name-1.conf 
fastcgi_pass 192.168.200.10:9501;
johnyc20@centos:~$

 How to comment lines

  Let's comment all lines starting with "fastcgi_pass":
johnyc20@centos:~$ sed -i '/fastcgi_pass/s/^/#/' *.conf
And check if it worked:       
johnyc20@centos:~$ cat site-name-1.conf                
#fastcgi_pass 192.168.200.10:9501;
johnyc20@centos:~$ 

 How to insert lines

I am gonna insert the "[tab]fastcgi_pass          127.0.0.1:9501;" after the line starting with "#fastcgi_pass":

johnyc20@centos:~$ sed -i '/#fastcgi_pass/a \\tfastcgi_pass          127.0.0.1:9501;' *.conf
Checking if the new inserted line is there:

johnyc20@centos:~$ cat site-name-1.conf
#fastcgi_pass 192.168.200.10:9501;
        fastcgi_pass          127.0.0.1:9501;

How to delete lines

Now we will delete the lines starting with the "#fastcgi_pass" string:
johnyc20@centos:~$ sed -i '/#fastcgi_pass/d' *.conf
Checking if it worked:

johnyc20@centos:~$ cat site-name-1.conf               
        fastcgi_pass          127.0.0.1:9501;

Short summary of sed arguments and options

Arguments

  • no argument - just output the way that the file will look like, applying the sed changes
  • -i - inline edit, changing the actual file
  • -ie - inline edit, but keep a copy of the original with an "e" on the end of file
's/match/string/g'
  • s - substitute
  • match - the string that will be substitute
  • string - the string that will replace the 'match' string
  • g - global, apply the action for all occurrences of the 'match' string, by default it will match just first occurrence for each line
  • use / or # as separator between fields both 's/match/string/g' and 's#match#string#g' are valid; if the string already has a '/', the use the '#' separator.

Comments

Post a Comment

Popular posts from this blog

JunOS - mount USB stick

By -
There are times when you have to use a USB stick to copy to or from it files to a Juniper router/switch. Fortunetly it will not be to very frequently, but there will be a time when you will need it. JunOS being a UNIX( FreeBSD ) based OS, you have to identify and mount the device attached. First, you have to check the device name, so you can identify it on /dev: smocanu@j2350> show log messages | match mass May 16 15:25:45  j2350 /kernel: umass0: Kingston DataTraveler SE9, rev 2.00/1.00, addr 2 May 16 15:25:46  j2350 /kernel: da0 at umass-sim0 bus 0 target 0 lun 0 Then you have to verify the partition number, usually it is the first one: smocanu@j2350> start shell % ls -las /dev/da0 /dev/da0 /dev/da0s1 Now you can mount your USB stick: % mkdir /var/tmp/usb % mount_msdosfs /dev/da0s1 /var/tmp/usb Once your USB stick is mounted, you can copy to/from it files, using /var/tmp/usb path. Do not forget that you have to umount the USB storage, before removing it fr
Read more

SSH: Generating and using SSH keys

By -
Image
One of the preferred method of authentication on network devices is using SSH with a SSH public key. In order to be able to authenticate yourself, you have to generate a SSH key pair. A SSH key is composed of two parts, one private key (which should remain "private" and also should be password protected ) and one public key which should be installed on the SSH server in order to authenticate you. SSH client on Linux: In order to generate a SSH key pair on Linux, you will need to use "ssh-keygen" tool, which is a part of the "openssh-client" package on Debian-like operating systems: smocanu@debian7:~$ dpkg -S $(which ssh-keygen) openssh-client: /usr/bin/ssh-keygen or a part of "openssh" package on RedHat-like operating systems: smocanu@centos6 ~$ rpm -qf $(which ssh-keygen) openssh-5.3p1-94.el6.x86_64 The generating process is quite simple, just run the "ssh-keygen" command, and it will ask you the location of the new key
Read more

Linux: Versioning the /etc/ configurations using 'etckeeper'

By -
One of the most important things for an administrator is to keep track of the changes. The preferred  way to do that is to use a configuration management system , like ' cfengine ', ' puppet ', ' chef ', ' ansible ' or you named it. Even so, you should still be able to see what exactly was changed on you system and when. I'm using ' etckeeper '  to do that and I'm thinking that it is very useful. The most important Unix distributions are including this software and you can install it by using your favorite package manager. On CentOS6, you can run: [root@centos6 ~]# yum install etckeeper After installing, at least on CentOS, you have to initialize the repository, by running: [root@centos6 ~]# etckeeper init Initialized empty Git repository in /etc/.git/ The last thing is to commit these changes, that means that you will add the entire /etc directory to the repository: [root@centos6 ~]# etckeeper commit -m "initial com
Read more