Tag Archives: tips

CURL usage

There are some powerful features of curl you did not know before

Used curl to grab all headers with ‘-H’

$ curl -I "http://iallex.com:8080"
HTTP/1.1 200 OK
X-Powered-By: PHP/5.5.1
X-Pingback: http://iallex.com:8080/xmlrpc.php
Content-Type: text/html; charset=UTF-8
Date: Mon, 24 Mar 2014 02:04:01 GMT
Server: lighttpd/1.4.34

Send a header to web server with ‘-H’

  • The curl command supports -H or –header option to pass extra HTTP header to use when getting a web page from a web server.
  • This option can be used multiple times to add/replace/remove multiple headers, the syntax is:

curl -H 'HEADER-1' -H 'HEADER-2' ... <URL>

E.g Check if 10.237.110.22:8088 apache node is working or not

curl -I -H 'Host: iallex.com' 'http://10.237.110.22:8080/'

Checking gzip/deflate server responses with curl

Curl provides a simple tool for checking server responses.

First, a few curl arguments that will come in handy:

-s, --silent prevents curl from showing progress meter

-w, --write-out 'size_download=%{size_download}\n' instructs curl to print out the download size

-o, --output instructs curl to throw away the output, sending it to /dev/null

Using these arguments, we can make a simple request for a path on the server:

curl -s -w "size_download=%{size_download}\n" -o /dev/null \
     "http://code.jquery.com/jquery-2.1.0.min.js"
size_download=83615

Here, you can the response was 83615 bytes. Next up, lets make the same
request, this time adding the Accept-Encoding header to ask for compressed
content.

curl -s -w "size_download=%{size_download}\n" -o /dev/null \
     -H "Accept-Encoding: gzip,deflate" \
     "http://code.jquery.com/jquery-2.1.0.min.js"
size_download=34151

Nice! This downloaded only 34151 bytes of data, so it the data is definitely
being compressed. Up next, lets try making the request a third time, now
making the request a HTTP1.0 request.

curl -s -w "size_download=%{size_download}\n" -o /dev/null \
     -H "Accept-Encoding: gzip,deflate" \
     --http1.0
     "http://code.jquery.com/jquery-2.1.0.min.js"
size_download=83615

This time, response same as first request, which makes sense when using Nginx
with the [gzip_http_version](http://wiki.nginx.org/NginxHttpGzipModule#gzip_ht
tp_version) set to 1.1.

Specify the user name and password to use for server authentication

Reference from manual

-u, –user

Specify the user name and password to use for server authentication. Overrides -n,
–netrc and –netrc-optional.

If you simply specify the user name, curl will prompt for a password.

The user name and passwords are split up on the first colon, which makes it
impossible to use a colon in the user name with this option. The password can, still.

curl -u allex:d9e871f "http://10.200.11.128:7890?do=syncx"

References:

Git daily tips

Retrieve a single file from specific revision in Git

git show somebranch:path/to/your/file

We can also do multiple files and have them concatenated: git show branchA~10:fileA branchB^^:fileB

NOTE:

If you want to get the file in the local directory (revert just one file) you can checkout: git checkout somebranch^^^ -- path/to/file

Remove local (untracked) files from my current Git branch

git-clean

git clean -f -d

If needed to remove untracked files from particular subdirectory:

git clean -f {dir_path}

And combined way to delete untracked dir/files and ignored files:

git clean -fxd {dir_path}

Git status give the output in an easy-to-parse format for scripts.

git status --porcelain

Pull with rebase instead of merge

$ git pull --rebase

# e.g. if on branch "master": performs a `git fetch origin`,
# then `git rebase origin/master`

When across merge commits, we’re get a [merge commits] with a message reading something like Merge branch 'master' of 'origin/master'.So we can avoid the unnecessary micro-merges on regular git pull by --rebase options.

Rebasing ensures that the commits are always re-applied so that the history stays linear. git will move your local commit aside, synchronise with the remote and then try to apply your commits from the new state.

You can configure certain branches to always do this without the --rebase flag:

# make `git pull` on master always use rebase
$ git config branch.master.rebase true

You can also set up a global option to set the last property for every new tracked branch:

# setup rebase for every tracking branch
$ git config --global branch.autosetuprebase always

You can configure all of pull with rebase option: git config --global pull.rebase true, and use git pull --no-rebase to disable this feature.

I usually use a fetch/rebase combination so my current (local) work stays at the top:

git fetch
git rebase origin/release-1.0.0

Git get specific branch head version.

git ls-remote --heads git@git.n.xiaomi.com:yp-develweb/devel-web-home.git release-1.0.0 | awk '{print $1}' | cut -c1-10

NOTE:

if in git shell context, can use these commonds:

last_commit=$(git rev-parse --short HEAD)
last_commit=$(git log --pretty=format:'%h' -n 1)

Stashing changes in a dirty working directory away

Stashing is a great way to pause what you’re currently working on and come back to it later.

read from Stashing your changes
Normally, we can use different local branches git branch xxx for jobs. But this may causes lot of unexpected logs in commits.
Use git stash when you want to record the current state of the working directory and the index, but want to go back to a clean working directory. The command saves your local modifications away and reverts the working directory to match the HEAD commit.

For some issues, we need git rebase to HEAD and fix some bugs:

git stash -u save current state, with git clean, leaving the working directory in a very clean state.

do some bugfixes and commit …

git stash apply Like pop, but do not remove the state from the stash list. to restore previous jobs states.

As of git 1.7.7, git stash accepts the –include-untracked option (or short-hand -u). To include untracked files in your stash, use either of the following commands:

git stash --include-untracked
git stash -u

Tips: Force git stash to overwrite added files

Use git checkout instead of git stash apply:

git checkout stash -- .
git commit

This will restore all the files to their stashed version.

If there are changes to other files in the working directory that should be kept, here is a less heavy-handed alternative:

git merge --squash --strategy-option=theirs stash

Note: for more details about stash, please view docs: git stash

Git Tags

# Create a new tag from your current HEAD (i.e. the HEAD of your current branch)
git tag <TAGNAME>

git tag <TAGNAME> <COMMIT> you can even specify which commit to use for creating the tag.
Regardless, a tag is still simply a “pointer” to a certain commit (not a branch).

Rename a Git tag

# build an alias of the old tag name:
git tag new_tag_name old_tag_name

# Then you need to delete the old one locally:
git tag -d old_tag_name

# delete the tag on you remote location(s)
# can be simplified to `git push origin :old_tag_name`
git push origin :refs/tags/old_tag_name

# add your new tag to the remote location
git push origin --tags

Other useful commands

Bypassing the git hooks by -n, like git commit -n [...]

-n, –no-verify
This option bypasses the pre-commit and commit-msg hooks.

# rebase these commit since dd61ab32 from HEAD
git rebase -i dd61ab32^
# Deleting the last commit
git push mathnet +dd61ab32^:master

Where git interprets x^ as the parent of x and + as a forced non-fastforward push.
This command same as:

# do it in two simpler steps: First reset the branch to the parent of the current commit, then force-push it to the remote.
git reset HEAD^ --hard
git push mathnet -f

Using pre-push git hook to runs unit tests on every push

read from git pre-push
Below is an example pre-push script that let’s us specify a branch to ‘protect’ so that our tests will only run if there are commits to push and we are on ‘master’. Also because pre-push will execute regardless of if there are commits to push or not, the script ensures we don’t fire off a lengthy test command, only to find out we actually didn’t need to.

#!/bin/bash 

CMD="ls -l" # Command that runs your tests
protected_branch='master'

# Check if we actually have commits to push
commits=`git log @{u}..`
if [ -z "$commits" ]; then
    exit 0
fi

current_branch=$(git symbolic-ref HEAD | sed -e 's,.*/\(.*\),\1,')

if [[ $current_branch = $protected_branch ]]; then
    $CMD
    RESULT=$?
    if [ $RESULT -ne 0 ]; then 
        echo "failed $CMD"
        exit 1
    fi
fi
exit 0

For more about git hooks we can read from the manual of githooks.

More git hooks articles:

sudo & redirect output

Like normally command append host sudo echo 127.0.0.1 local.host > /etc/hosts, but we’ll get the Permission denied error message.

sudo testparm /etc/samba/smb.conf.master > /etc/samba/smb.conf
bash: /etc/samba/smb.conf: Permission denied

Then how to sudo and redirect output to a file?

One solution is to use sh -c option:

sudo sh -c "testparm /etc/samba/smb.conf.master > /etc/samba/smb.conf"

Or you could use a tee command like this:

testparm /etc/samba/smb.conf.master | sudo tee /etc/samba/smb.conf

More about tee usage

http://linux.101hacks.com/unix/tee-command-examples/

Git Basics: diff commit and merge branches

GIT DIFF

Use git diff show the difference between commits.

git co master
git diff 0da94be..59ff30c > /tmp/my.patch
# Checkout new branch: (this will not reset your work)
git co branches-dev git apply /tmp/my.patch

Diff a single file by different commits:

git diff 5bb72a8 HEAD -- twentytwelve/style.cs

Compare files from two different branches:

# You can do this: `git diff branch1:file branch2:file`
git diff master:twentytwelve/style.css branche-dev:twentytwelve/style.css

Git – Create a branch with current changes

Just use:

git checkout -b topic/newbranch

Any uncommitted work will be taken along to the new branch.


GIT Merge

Merge some (not all) files from one Git branch to another.

As it turns out, we’re trying too hard. Our good friend git checkout is the right tool for the job.

git checkout <branch> <paths>...

We can simply give git checkout the name of the feature branch and the paths to the specific files that we want to add to our master branch.

# merge a single commit from another branch
git cherry-pick <commit_sha1>

# merge without auto commit
git merge v1.0 --no-commit --no-ff

Syncing a fork

Merge the changes from upstream/master into your local master branch. This brings your fork’s master branch into sync with the upstream repository, without losing your local changes.
We can configure git to sync your fork with the original repository git remote add upstream <ORIGINAL_REPOSITORY.git>

git merge upstream/master
# Updating a422352..5fdff0f
# Fast-forward
#  README                    |    9 -------
#  README.md                 |    7 ++++++
#  2 files changed, 7 insertions(+), 9 deletions(-)
#  delete mode 100644 README
#  create mode 100644 README.md

Git merge “accept theirs” or “accept mine” options

git checkout --ours -- <filename>
git checkout --theirs -- <filename>

Resolving the conflict

Just modify the files conflicting. You would need to call git add to mark the conflict as resolved:

$ git add file_with_resolved_conflicts

This will also add the file to the index. After you repeat the same process for other conflicting files, you can safely commit your changes using git commit command:

git commit -m 'Merged with branch src-branch and resolved the conflicts.'

ref: http://softwarecave.org/2014/03/03/git-how-to-resolve-merge-conflicts/


Helper commands

  • use git difftool to manally merge files: git config merge.tool vimdiff
  • undo current changess: git reset --hard origin/master

Reference Links

Adding And Removing Remote Branches – Git Branch

git squashing commits with rebase

git rebase – Forward-port local commits to the updated upstream head

We can use git rebase --interactive (or -i) mode to squash multiple snapit commit.

git rebase -i HEAD~2
pick b76d157 b
pick a931ac7 c

Changing b’s pick to squash will result in the error you saw, but if instead you squash c into b by changing the text to:

pick b76d157 b
s a931ac7 c

For more details reference:

Get absolute path from a shell path

While coding shell script, normally get the current directory of the script file like:

sh_dir=`cd -P — “$(dirname “$0″)” && pwd -P`

But, if running by source like source foo.sh, we may got the unexpected result.

Here is a safer and more readable way to do this job:

# get current script directory
DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )

# get current executing script full path
sh_path=$( unset CDPATH && cd "$(dirname "$0")" && echo $PWD/$(basename "$0") )

Notes:

  • If $0 is a bare filename with no preceding path, the original script will fail but the one given here will work. (Not a problem with $0 but could be in other applications.)
  • Either approach will fail if the path to the file doesn’t actually exist. (Not a problem with $0, but could be in other applications.)
  • The unset is essential if your user may have CDPATH set.
  • Unlike readlink -f or realpath, this will work on non-Linux versions of Unix (e.g., Mac OS X).
DIR=$(cd `dirname "${BASH_SOURCE[0]}"` && pwd)/

Using ${BASH_SOURCE[0]} instead of $0 produces the same behaviour
regardless of whether the script is invoked as

name.sh or source name.sh

For more details about difference in linux between ‘source’ and ‘sh’

Enable Root User ( Super User ) in Ubuntu

Question: I’m unable to do su – on Ubuntu. It says “su: Authentication failure”. How do I fix it? Also, is it possible for me to login to Ubuntu using root account directly?

Answer: Let us address these two question one by one.

Warning: Enabling root is not recommended. If possible, you should always try to perform all administrative tasks using sudo.

Question 1: I’m unable to login using su command. How to fix this?

By default, root account password is locked in Ubuntu. So, when you do su -, you’ll get Authentication failure error message as shown below.

$ su -
Password:
su: Authentication failure

Enable super user account password on Ubuntu

First, set a password for root user as shown below.

$ sudo passwd root
[sudo] password for ramesh:
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

Now with the new password you can login as super user with su command

$ su -
Password:
#

Disable super user account password on Ubuntu

Later if you don’t want to use su anymore, you can lock the root user password using one of the methods shown below

$ sudo passwd -l root

( or )

$ sudo usermod -p '!' root

Step 2: In the Security tab, click on the check box “Allow local system administrator” as shown below. After this change, reboot the system and login directly using root account.

Fix adb devices android adb device not found

ISSUES: 小米手机 usb 连接 adb debug 没反应.(adb devices 提示找不到设备或是没有权限)

$ adb devices get the following messages:

- waiting for device -

adb kill-server
adb devices
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
List of devices attached

Detect your device USB ID by lsusb

idVendor:idProduct

lsusb
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 003 Device 002: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 003 Device 027: ID 2717:9039  
Bus 003 Device 003: ID 04f2:b327 Chicony Electronics Co., Ltd 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
.....

If cannot identity by name string, we can diff the results by plugin USB on/off of your device.

Set up your system to detect your device

The fix is mentioned on the setup page I mentioned previously.

Create a file called something like /etc/udev/rules.d/99-android.rules that contains rules telling udev to make the device node world-writable when a matching device is found.

sudo vi /etc/udev/rules.d/99-android.rules

SUBSYSTEMS=="usb", ATTRS{idVendor}=="2717", ATTRS{idProduct}=="9039", MODE="0666", OWNER="allex"

For simplicity, you could specify “*” for ATTRS so that all devices can be used

SUBSYSTEMS=="usb", ATTRS{idVendor}=="*", ATTRS{idProduct}=="*", MODE="0666", OWNER="your_name"

Manually update vendor ID to ~/.android/adb_usb.ini

If adb still cannot detect your device. the solution is that the ADB daemon needs also your USB device vendor ID (VID)

mkdir ~/.android/
echo 0x2717 > ~/.android/adb_usb.ini # 其中 0x2717 是通过 lsusb 得到的 VendorId

For more VID please see also at adbusb.ini and the Linux USB listing at http://www.linux-usb.org/usb.ids

Restart udev and adb services

sudo restart udev
adb kill-server && adb start-server
    
adb devices
List of devices attached 
66c95e6 device

DONE!

Enjoy android develops with adb logcat

Apache prevent web browsers from caching

In order to get the latest effections, Front-end develop need to refresh the web page frequently.

there some solution for fix page cache problem:

Ensure the request url is unique.

by adding random seed to URL. such as

<script src="http://xx.com/foo/path.js?v=${version}"></script>

But this case need some server programe to generic the html page resouces references dynamic.

The other one and the more simple way is prevent Web browsers by customize HTTP Headers.

Setting a short cache time

By asking the Web browser to only cache the file for a very short length of time, you can usually avoid the problem.

We support the Apache Web server “mod_expires” feature. To use this to set the cache time to just one second, add these lines to a .htaccess file:

ExpiresActive On
ExpiresDefault A1
Header append Cache-Control must-revalidate

That’s all it takes. The next time the visitor views the file, more than a second will have passed, so the browser won’t use the outdated cached copy.

Controlling which files are affected

Disabling browser caching will slow down your site and increase the amount of bandwidth your site uses, because repeat visitors will always connect to your site to re-download files they would otherwise cache.

To minimize that impact, you might want to prevent caching of files that change often (such as HTML files), while allowing normal caching of files that don’t (such as JPEG files).

To do this, include the .htaccess lines in a <FilesMatch> directive. For example, these lines will prevent caching of filenames ending in “.htm” or “.html”, while allowing normal caching of JPEG files:

<FilesMatch "\.(htm|html|js|css)$">
  ExpiresActive On
  ExpiresDefault A1
  Header append Cache-Control must-revalidate
</FilesMatch>

(An alternate way of doing this is to use the Apache “ExpiresByType text/html” command, but that doesn’t allow you to add the “Cache-Control: must-revalidate” header for only those pages.)

For more info about Apache headers see also: Manipulating HTTP Headers with htaccess.

Vim get the current cursor char ascii code and replace it

In command mode press the keys ga (like get ascii) or :as / :ascii on the command line to find out the ascii code of the character where the cursor is. This displays the value of the current character in decimal, hex and octal. (Think “get ascii.”)

To do a search and replace with an hex code

:%s/\%x0d.*$//d

Being 0d the ascii character found with ga

Seeing ASCII/Unicode values in the status line

You can also add options to the statusline option to show the ASCII/Unicode value of the character under the cursor.

:set statusline=%<%f%h%m%r%=%b\ 0x%B\ \ %l,%c%V\ %P

It’s the %b\ 0x%B that does the trick. Note that you may need to :set laststatus=2 so that the status line is visible in with only one window showing.

See also at http://vim.wikia.com/