In this post, we will see how to establish a passwordless SSH connection between two Linux machines. This can be done by using the Public and Private keys. Once the keys are set up, the authentication will be done using these keys instead of the password. Let us consider there are two Linux machines/servers ServerA… Read More »
In this blog, we will see how to archive/delete a file in HDFS if it is n days older. We can use this to check for any number of days. For example, let us say that we need to monitor an HDFS folder and delete the files when they become 7 days older.
Looping through the list is a very useful and much-needed function for every scripting and programming language. The most common looping method is the programming world is the for loop. In this blog, we will see how to iterate over strings separated by a delimiter in a shell script. For loop Syntax Let us see… Read More »
In the ETL world, it is always a need for processing the delimited files. While reading a delimited file can be done through all the scripts and programming languages, we will see how this can be done in a shell script using the cut command Syntax cut -d “|” => This is to tell the… Read More »
Command line arguments are essential for any programming or scripting language to have a control of the the input parameters. Passing arguments to the shell script can be done in multiple ways. In this blog, we will see the two ways of passing arguments to a shell script Option 1 – Using getops This method… Read More »
We often need the email facility in the shell script as most of the batch jobs in production are running in Linux based machines only. In case of issues, the jobs or scheduler needs to send failure notifications to the support DL. At times we may need the failure log as an attachment in the… Read More »
We can delete the first line in a file using ‘sed’ command in the shell script. Let us see with the help of a test file called ‘sample.txt’. Here are the contents of the text file. Option 1: Delete and save in a new file In this option, a new file will be created with… Read More »
We will see the usage of grep command in different cases. The command can be used in different ways based on the need to search a string or substring in a file or in multiple files in a folder. The grep command stands for “global regular expression print” General format To see the full syntax… Read More »