#1 Exclude a sub directory while compressing a folder
While compressing a folder we often need to exclude some sub directories. I used this most of the times I work with python virtual environment.
tar -pczf flask-vehicle-app.tar.gz flask-works/ –exclude “flask-works/env”
Make sure that quotes are there for the folder name to be excluded…
An environment variable like JAVA_HOME can be set in different ways in linux systems.
#1 . If we set environment variables in .bashrc file found in each users home directory, it will be accessible only for that user.
~/.bashrc ——> specific to a particular user.
Setting environment variable for all users.
If the environment variable to be available for all users,
#2. it can be set in /etc/environment
For instance let see how JAVA_HOME can be set,
Open this file in vim or any editor of your choice, and add following line in it.
JAVA_HOME=”/usr/lib/jvm/java-8-openjdk-amd64″.
This sets JAVA_HOME for default open JDK. Please note that location is specified in quotes,
Then save the file and run the command, to have these these changes in effect immediately.
source /etc/environment
But this is not a recommended method.
#3. Setting via /etc/profile
vim /etc/profile (open the this file in vim )
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export PATH=$JAVA_HOME/bin:$PATH
Then save the file and run following command.
source /etc/profile
#4. Setting via /etc/profile.d
This can be viewed as an extension to the above step. Having all application specific environment file in /etc/profile makes the maintenance a hassle. As a solution to this, variables will be specified as a separate module inside the directory /etc/profile.d/. These modules are shell scripts and should end with .sh extension, otherwise system is not going to recognize it. You can read further on this from here.
I think this would be clear, with the following example.
In order to set JAVA_HOME bythis approach, first create a java_dev.sh file inside /etc/profile.d (The file should end with .sh extension ).
Then open this file using vim or any editor
vim /etc/profile.d/java_dev.sh
Then add following lines ( This JAVA_HOME location could be different for others)
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export PATH=${JAVA_HOME}/bin:${PATH}
Then save the file and run the following command.
source /etc/profile
You can also consider reading this
When we copy a certain folder to a remote machine using scp, we might want to exclude some sub folders or files. The following command worked for me in those kind of situations.
scp !(bin) root@target-machine:/opt/your-folder-name/
In this case, it copies everything except the bin folder from the current working directory to /opt/your-folder-name/ folder in target-machine
Extract data as csv from mysql table
From the termial run the following command (you may need to unformat this if you are copying this command, the quotes may appear in different format),
mysql -u root -pspace123 -e “SELECT * from employee” mydb > asd.csv The above command export data from the table employee of the database mydb into a csv file called asd.csv . By default the delimiter would be tab in this csv file and this can be overidden. Also note that there no space after -p. The password is expected be written along with -p attribute unlike what we do for -u. Otherwise error will be thrown.
asd.csv empno ename
1 ram
1000 Jeena
In this, the header row can be omitted by specifying -N parameter along with the previous command
mysql -N -u root -pspace123 -e “SELECT * from employee” mydb > asd.csv
asd.csv
1 ram
1000 Jeena
Upload csv into mysql table For this, the only requirement is that we need to have a table in the database with the same and structure as that of the csv file we are going to upload. In my case I have an employee.csv file and table with same name employee and same structure.
employee.csv
1;ram
1000;Jeena
Now type the following command from the terminal,
mysqlimport -u root -pspace123 –fields-terminated-by=’;’ –local mydb employee.csv
This would upload this csv file to the table employee and that can be viewed from mysql console
mysql> select * from employee;
Upload csv file into Remote Table
mysqlimport -h remote-host-name –port 3306 -u username -ppassword –fields-terminated-by=’;’ –local remote-db-name filename.csv-on-local-machine
mysqlimport -h example.com –port 3306 -u testUser -ptestUser123 –fields-terminated-by=’;’ –local demo test.csv
For importing a csv file into mysql table, we need to execute following command:
LOAD DATA INFILE ‘/tmp/t.csv’ INTO TABLE table1 FIELDS TERMINATED BY ‘,’ ENCLOSED BY ‘”‘ LINES TERMINATED BY ‘\n’;
Sometimes we may face some file not found exceptions and errors and it may persist even after giving full permissions. I got an error which said t.csv file is missing even though I have place it at correct location. The fix for this issue is to change the ownwership of t.csv to mysql using the following command.
chown mysql:mysql t.csv
DATE filed and CSV imports
I had a table table2:
create table table2(ID INT,title INT NULL,dt DATETIME NULL);
t.csv :
1, 3,10/12/2000
1000, 1223,12/12/2014
And when I executed the previopus load command, I got incorrect values in mysql table table2 and which was as follows:
ID title dt
+——+——-+———————+
1 3 0000-00-00 00:00:00
1000 1223 0000-00-00 00:00:00
In this case, we need to modify the date filed while uploading to database. This can be done using the following command:
LOAD DATA INFILE ‘/tmp/t.csv’ INTO TABLE table2 FIELDS TERMINATED BY ‘,’ ENCLOSED BY ‘”‘ LINES TERMINATED BY ‘\n’ (ID, title, @var1) SET dt = STR_TO_DATE(@var1, ‘%m/%d/%Y’);
ID title dt
+——+——-+———————+
1 3 2000-10-12
1000 1223 2014-12-12
And if there are multiple fields which are to be modified,you can specify them in a comma separated way like,
LOAD DATA LOCAL INFILE '/path/to/csv/file.csv'
INTO TABLE mytable
LINES TERMINATED BY '\n'
(id, task, hoursWorked, @var1, @var2)
SET begindate = STR_TO_DATE(@var1, '%m/%d/%Y'),
enddate = STR_TO_DATE(@var2, '%m/%d/%Y');When I tried to run sudo apt-get update, I came across something like this.
Ign http://in.archive.ubuntu.com precise InRelease
Err https://archive.cloudera.com precise-cm5.5.1/contrib amd64 Packages
Couldn’t resolve host ‘archive.cloudera.com’
Err http://archive.cloudera.com precise-cm5/contrib Sources
Something wicked happened resolving ‘archive.cloudera.com:http’ (-5 – No address associated with hostname). Ign http://in.archive.ubuntu.com precise-updates InRelease
Err http://archive.cloudera.com precise-cm5/contrib amd64 Packages
Something wicked happened resolving ‘archive.cloudera.com:http’ (-5 – No address associated with hostname)
Ign http://in.archive.ubuntu.com precise-backports InRelease
Err http://archive.cloudera.com precise-cm5/contrib Translation-en_IN
Something wicked happened resolving ‘archive.cloudera.com:http’ (-5 – No address associated with hostname)
This was a bit scary for me as it was for the first time I was facing something like this. And I even tried adding hostname of ubuntu server and its ip address directly in my /etc/hosts file. and further to my frustration it did not work. But The actual fix that worked for me is as follows:
echo “nameserver 8.8.8.8” | sudo tee /etc/resolv.conf > /dev/null sudo apt-get update
Then apt-get update worked perfectly.
cp -a /source/. /dest/
The -a option is an improved recursive option (which actually stands for cp --archive), that preserve all file attributes, and also preserve symlinks.
The . at end of the source path is a specific cp syntax that allow to copy all files and folders, included hidden ones.