Use SVN+SSH under Windows and Linux

This post is a simple write-up of the process and problem during my experience. Note: Please replace username to what you need in this post!

Under Linux (Ubuntu here)

1. Create public and private key under Linux

ssh-keygen -t rsa -b 4096 -f <username>

You will see two new file under the directory.

2. You need to send the username.pub file to the server administrator and you keep the private key file username.

3. Test SSH

ssh -i username username@host(or IP)

Note: the first username is your private key file and the second one is the username for login.

4. Test SVN

svn co svn+ssh://username@host/path_to_repository

Under Windows (Win 7)

1. Get the copy of the private key file

2. Convert the key into Putty format. Start PuTTYgen and use Conversions -> Import key Browse to your file keyfile which you got from the server. Finally click on Save private key and save the file as keyfile.PPK.

3. Use this new key file for SSH. Start putty and on the Session tab set the hostname to the name or IP address of your server, the protocol to SSH and save the session as SvnConnection or whatever name you prefer. On the SSH tab set the preferred SSH protocol version to 2 and from Auth set the full path to the .PPK private key file you converted earlier. Go back to the Sessions tab and hit the Save button. You will now see SvnConnection in the list of saved sessions.  Click on Open and you should see a telnet style login prompt.

4. Test TortoiesSVN. One way to simplify the URL in TortoiseSVN is to set the user inside the PuTTY session. For this you have to load your already defined session SvnConnection in PuTTY and in the Connection tab set Auto login user to the user name, e.g. svnuser. Save your PuTTY session as before and try the following URL inside TortoiseSVN: svn+ssh://SvnConnection/repos

5. Check out the SVN use the same URL.

Reference: http://tortoisesvn.net/ssh_howto.html

Useful Linux Commands

sort

sort -T . (use currently directory instead system tmp)

sort -t ”     ” -k3,3nr -k4,4n  input.txt > output.txt (sort the third numerical column in descending order and the forth numerical column in ascending order)

awk

awk '{s+=$1} END {print s}' mydatafile (sum number, one per line)

du

du –si * (show total file usage in human readable way)

ln

ln -f -s target_directory symbolic_link_directory (This will update existing link without removing old one.)

Uniq command

After sorting a file you will often find that some duplicate data, or you may be given various lists that need deduping. sort and uniq will quickly and easily remove duplicates, lsit only the dupilcates or only the unique data.

sort myfile.txt | uniq

List only the unique lines: sort myfile.txt | uniq -u

List only the duplicate lines: sort myfile.txt | uniq -d

Get a count of the number of lines by adding the -c option.

sort myfile.txt | uniq -uc

sort myfile.txt | uniq -dc

Skip fields: uniq -f 3 mylogfile. this could be useful with log files to skip the time stamp data

Skip characters. uniq -s 30 myfile.txt. Skip the first 30 characters

Compare characters. uniq -w 30 myfile.txt. Compare the first 30 characters

Generate SSH Key

  • ssh-keygen -t rsa -b 4096

Read RPM

  • rpm -qlp rpm-name

Search directory

  • find / -name directory-name -type d

Change to lower case

  • $ tr '[:upper:]' '[:lower:]' < input.txt > output.txt

Screen command

  • Ctrl+a c (new window)
  • Ctrl+a n (select the window)
  • Ctrl+a d (detach the session)
  • screen -ls (list session)
  • screen -r session name (attach to a detach session)
  • screen -rD session name (attach to an active session)

Read RPM content

  • rpm -qlp rpm-name

Find a directory or a file

  • find / -name directory-name -type d
  • find / -name file-name -type f

How do I extract a gz file?

  • $ gunzip file.gz
  • $ gzip -d file.gz

How do I extract a tar.gz or .tgz file?

  • $ gunzip < file.tar.gz | tar xvf –
  • $ gunzip < file.tgz | tar xvf –
  • $ tar xvzf file.tar.gz
  • $ tar xvzf file.tgz

Processing the delimited files

  • cut -d: -f1,3  /etc/passwd
  • awk -F’:’   ‘{print $1,$3}’   /etc/passwd

Copy files from remote machine

  • rcp -rp username@hostname(or IP):/path localPath (r: recursive, p: preserve)
  • scp is similar and based on SSH

Check HDFS system

  • bin/hadoop dfsadmin -report
    Configured Capacity: 603258257408 (561.83 GB)
    Present Capacity: 455640743936 (424.35 GB)
    DFS Remaining: 455623647232 (424.33 GB)
    DFS Used: 17096704 (16.3 MB)
    DFS Used%: 0%
    Under replicated blocks: 0
    Blocks with corrupt replicas: 0
    Missing blocks: 0
    
    -------------------------------------------------
    Datanodes available: 2 (2 total, 0 dead)
    
    Name: 192.168.241.22:50010
    Decommission Status : Normal
    Configured Capacity: 452167245824 (421.11 GB)
    DFS Used: 8548352 (8.15 MB)
    Non DFS Used: 30800699392 (28.69 GB)
    DFS Remaining: 421357998080(392.42 GB)
    DFS Used%: 0%
    DFS Remaining%: 93.19%
    Last contact: Thu Apr 21 09:59:01 PDT 2011
    
    Name: 192.168.241.23:50010
    Decommission Status : Normal
    Configured Capacity: 151091011584 (140.71 GB)
    DFS Used: 8548352 (8.15 MB)
    Non DFS Used: 116816814080 (108.79 GB)
    DFS Remaining: 34265649152(31.91 GB)
    DFS Used%: 0.01%
    DFS Remaining%: 22.68%
    Last contact: Thu Apr 21 09:59:02 PDT 2011
    

Others