Category Archives: develop

develop

Git daily tips

Retrieve a single file from specific revision in Git

git show somebranch:path/to/your/file

We can also do multiple files and have them concatenated: git show branchA~10:fileA branchB^^:fileB

NOTE:

If you want to get the file in the local directory (revert just one file) you can checkout: git checkout somebranch^^^ -- path/to/file

Remove local (untracked) files from my current Git branch

git-clean

git clean -f -d

If needed to remove untracked files from particular subdirectory:

git clean -f {dir_path}

And combined way to delete untracked dir/files and ignored files:

git clean -fxd {dir_path}

Git status give the output in an easy-to-parse format for scripts.

git status --porcelain

Pull with rebase instead of merge

$ git pull --rebase

# e.g. if on branch "master": performs a `git fetch origin`,
# then `git rebase origin/master`

When across merge commits, we’re get a [merge commits] with a message reading something like Merge branch 'master' of 'origin/master'.So we can avoid the unnecessary micro-merges on regular git pull by --rebase options.

Rebasing ensures that the commits are always re-applied so that the history stays linear. git will move your local commit aside, synchronise with the remote and then try to apply your commits from the new state.

You can configure certain branches to always do this without the --rebase flag:

# make `git pull` on master always use rebase
$ git config branch.master.rebase true

You can also set up a global option to set the last property for every new tracked branch:

# setup rebase for every tracking branch
$ git config --global branch.autosetuprebase always

You can configure all of pull with rebase option: git config --global pull.rebase true, and use git pull --no-rebase to disable this feature.

I usually use a fetch/rebase combination so my current (local) work stays at the top:

git fetch
git rebase origin/release-1.0.0

Git get specific branch head version.

git ls-remote --heads git@git.n.xiaomi.com:yp-develweb/devel-web-home.git release-1.0.0 | awk '{print $1}' | cut -c1-10

NOTE:

if in git shell context, can use these commonds:

last_commit=$(git rev-parse --short HEAD)
last_commit=$(git log --pretty=format:'%h' -n 1)

Stashing changes in a dirty working directory away

Stashing is a great way to pause what you’re currently working on and come back to it later.

read from Stashing your changes
Normally, we can use different local branches git branch xxx for jobs. But this may causes lot of unexpected logs in commits.
Use git stash when you want to record the current state of the working directory and the index, but want to go back to a clean working directory. The command saves your local modifications away and reverts the working directory to match the HEAD commit.

For some issues, we need git rebase to HEAD and fix some bugs:

git stash -u save current state, with git clean, leaving the working directory in a very clean state.

do some bugfixes and commit …

git stash apply Like pop, but do not remove the state from the stash list. to restore previous jobs states.

As of git 1.7.7, git stash accepts the –include-untracked option (or short-hand -u). To include untracked files in your stash, use either of the following commands:

git stash --include-untracked
git stash -u

Tips: Force git stash to overwrite added files

Use git checkout instead of git stash apply:

git checkout stash -- .
git commit

This will restore all the files to their stashed version.

If there are changes to other files in the working directory that should be kept, here is a less heavy-handed alternative:

git merge --squash --strategy-option=theirs stash

Note: for more details about stash, please view docs: git stash

Git Tags

# Create a new tag from your current HEAD (i.e. the HEAD of your current branch)
git tag <TAGNAME>

git tag <TAGNAME> <COMMIT> you can even specify which commit to use for creating the tag.
Regardless, a tag is still simply a “pointer” to a certain commit (not a branch).

Rename a Git tag

# build an alias of the old tag name:
git tag new_tag_name old_tag_name

# Then you need to delete the old one locally:
git tag -d old_tag_name

# delete the tag on you remote location(s)
# can be simplified to `git push origin :old_tag_name`
git push origin :refs/tags/old_tag_name

# add your new tag to the remote location
git push origin --tags

Other useful commands

Bypassing the git hooks by -n, like git commit -n [...]

-n, –no-verify
This option bypasses the pre-commit and commit-msg hooks.

# rebase these commit since dd61ab32 from HEAD
git rebase -i dd61ab32^
# Deleting the last commit
git push mathnet +dd61ab32^:master

Where git interprets x^ as the parent of x and + as a forced non-fastforward push.
This command same as:

# do it in two simpler steps: First reset the branch to the parent of the current commit, then force-push it to the remote.
git reset HEAD^ --hard
git push mathnet -f

Using pre-push git hook to runs unit tests on every push

read from git pre-push
Below is an example pre-push script that let’s us specify a branch to ‘protect’ so that our tests will only run if there are commits to push and we are on ‘master’. Also because pre-push will execute regardless of if there are commits to push or not, the script ensures we don’t fire off a lengthy test command, only to find out we actually didn’t need to.

#!/bin/bash 

CMD="ls -l" # Command that runs your tests
protected_branch='master'

# Check if we actually have commits to push
commits=`git log @{u}..`
if [ -z "$commits" ]; then
    exit 0
fi

current_branch=$(git symbolic-ref HEAD | sed -e 's,.*/\(.*\),\1,')

if [[ $current_branch = $protected_branch ]]; then
    $CMD
    RESULT=$?
    if [ $RESULT -ne 0 ]; then 
        echo "failed $CMD"
        exit 1
    fi
fi
exit 0

For more about git hooks we can read from the manual of githooks.

More git hooks articles:

Make websafe colors

For Web colors plugins.

Convert a normal hex color to a websafe color.

var round = Math.round;
var floor = Math.floor;

function get_hex(dec) { return dec.toString(16); }
function get_dec(hex) { return parseInt('0x' + hex, 16); }

function rgb_to_hex(r, g, b) {
    var c1 = get_hex(floor(r / 16));
    var c2 = get_hex(floor(r % 16));
    var c3 = get_hex(floor(g / 16));
    var c4 = get_hex(floor(g % 16));
    var c5 = get_hex(floor(b / 16));
    var c6 = get_hex(floor(b % 16));
    return c1 + c2 + c3 + c4 + c5 + c6;
}
function hex_to_rgb(hex) {
    var i = 0, arr = [], c1, c2;
    while (i < 6) {
        c1 = get_dec(hex.substring(i, ++i));
        c2 = get_dec(hex.substring(i, ++i));
        arr.push((c1 * 16) + c2 * 1);
    }
    return arr;
}

function web_safe(r, g, b) {
    var t;
    t = r % 51; if (t > 25) { t = r + 51 - t; } else { t = r - t; }
    var c1 = get_hex(round(t / 17));
    t = g % 51; if (t > 25) { t = g + 51 - t; } else { t = g - t; }
    var c2 = get_hex(round(t / 17));
    t = b % 51; if (t > 25) { t = b + 51 - t; } else { t = b - t; }
    var c3 = get_hex(round(t / 17));
    return c1 + c1 + c2 + c2 + c3 + c3;
}

function get_safe_color(c) {
    if (c.charAt(0) === '#') c = c.substring(1);
    var rgb = hex_to_rgb(c), r = rgb[0], g = rgb[1], b = rgb[2];
    return '#' + web_safe(r, g, b);
}

console.log(get_safe_color('#1255FF')); // #0066FF

216 Web Safe Colors table see also http://websafecolors.info/

Convert thrift java bean to JSON object

Today we have a java bean generated by Thrift with schema like:

struct T {
   1: string address
}

Normally we can translate the java bean to a org.json.JSON object like:

...

T bean = new T();
JSONObject json = new JSONObject();
Map<String, Object> dataMap = org.apache.commons.beanutils.BeanUtils.describe(bean);
for (Entry<String, Object> entry : dataMap.entrySet()) {
    json.put(entry.getKey(), entry.getValue());
}
...

// or translate with JSONObject simple
...
JSONObject json = new JSONObject(bean);

But for beans which generated by thrift, Both we’ll get some no-used properties prefixes with set*:

{"address":"abc","setAddress":true}

……

After some googled, we are get some wiki API method, see also http://wiki.apache.org/thrift/ThriftUsageJava

/**
 * Convert the generic TBase<?, ?> entity to JSON object.
 *
 * @param tobj
 * @author Allex Wang
 * @return
 */
public JSONObject convertBeanToJSON(final TBase<?, ?> tobj) {
    TSerializer serializer = new TSerializer(new TSimpleJSONProtocol.Factory());
    try {
        String json = serializer.toString(tobj, "utf8");
        return new JSONObject(json);
    } catch (TException ex) {
        LOGGER.error("Convert TBase object to JSON fails: " + ex.getMessage());
    } catch (JSONException ex) {
    }
    return null;
}

Now we are get the JSON entity as expected:

{"address":"abc"}

Building a php extension using phpize

Run piece of PHP code in command line

php -r "echo(mcrypt_module_open('rijndael-256', '', 'ofb', ''));"

Then got an error message about Mcrypt:

Fatal error: Call to undefined function mcrypt_module_open() in Command line code on line 1

Install mcrypt for PHP53 on centos by phpize

cd php-5.5.9/ext/mcrypt/
phpize
aclocal
./configure
make
make install

Add the myrypt extension config /etc/php.ini containing:

extension=mcrypt.so

And check if the mcrypt module loaded or not by php -m | grep "mcrypt".

You may get some errors like the following

PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/php5/lib/php/extensions/no-debug-non-zts-20121212/mcrypt.so' - libmcrypt.so.4: cannot open shared object file: No such file or directory in Unknown on line 0

To fix it, enter yum install libmcrypt-devel

How to shrink/purge ibdata1 file in MySQL

That ibdata1 isn’t shrinking is a particularly annoying feature of MySQL. The ibdata1 file can´t actually be shrunk unless you delete all databases, remove the files and reload a dump.

But you can configure MySQL so that each table, including its indexes, is stored as a separate file. In that way ibdata1 will not grow as large. this is enabled by default as of version 5.6 of MySQL.

It was a while ago I did this. However, to setup your server to use separate files for each table you need to change my.cnf in order to enable this:

[mysqld]
innodb_file_per_table

http://dev.mysql.com/doc/refman/5.5/en/innodb-multiple-tablespaces.html

As you want to reclaim the space from ibdata1 you actually have to delete the file:

  1. Do a mysqldump of all databases, procedures, triggers etc except the mysql and performance_schema databases
  2. Drop all databases except the above 2 databases
  3. Stop mysql
  4. Delete ibdata1 and ib_log files
  5. Start mysql
  6. Restore from dump

When you start MySQL in step 5 the ibdata1 and ib_log files will be recreated.

Now you’re fit to go. When you create a new database for analysis, the tables will be located in separate ibd* files, not in ibdata1. As you usually drop the database soon after, the ibd* files will be deleted.

mysqldump –where option

The MySQL provides a great tool mysqdump to dump database. Its the official sql dump utitlity for MySQL database. It makes the life of dba so easy that he can backup and restore the database within just two commands. But sometimes due to the lacking of the infrastructure you can not dump and restore that easily. Specially when you are dealing with huge amount of data. Our database grows over time. Few hundred GBs are quite common. But if you run a software for long it might get in to Tera byte range. The problem starts when size is this huge.

Split the big tables by tables.

mysqldump db1 table1 table2 table3 > table1-3.sql

Split the big tables by rows with –where.

-w, --where=name    Dump only selected records. Quotes are mandatory.

If your table has auto column typed with auto_increment you can split by any number of chunks.

mysqldump --where "id%2=0" db1 table1 > table1_even.sql
mysqldump --where "id%2=1" db1 table1 > table4_odd.sql

limit clause.

# Dump the first 100000 rows from the table named table5 into the file dump.sql
# Use LIMIT [OFFSET, ] LIMIT So some thing like that:
mysqldump --where "1 LIMIT 10000" database1 table5 > dump.sql

Others. As the –where switch allows any sql condition you can use any criteria.

mysqldump --where "year(date) <= 2008" db1 payment > payment_prior_2009.sql

 Some useful options:

–skip-add-drop-table Avoid generate the DROP TABLE statements.
–skip-create-options Disable include all MySQL specific create options.
–replace Use REPLACE INTO instead of INSERT INTO.

php is NULL

I’ve some codes like:

// do something...

$json = json_decode($data);
if ($json == NULL) {
    $json = array("message" => "data null");
}
// do some out print
// ....

But, i hadn’t got the empty json result with message “data null” when $data is empty array();

After some google searches. references to http://php.net/manual/en/language.types.null.php

Note: empty array is converted to null by non-strict equal ‘==’ comparison. Use is_null() or ‘===’ if there is possible of getting empty array.

$a = array();
$a == null  <== return true
$a === null <== return false
is_null($a) <== return false

Fix adb devices android adb device not found

ISSUES: 小米手机 usb 连接 adb debug 没反应.(adb devices 提示找不到设备或是没有权限)

$ adb devices get the following messages:

- waiting for device -

adb kill-server
adb devices
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
List of devices attached

Detect your device USB ID by lsusb

idVendor:idProduct

lsusb
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 003 Device 002: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 003 Device 027: ID 2717:9039  
Bus 003 Device 003: ID 04f2:b327 Chicony Electronics Co., Ltd 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
.....

If cannot identity by name string, we can diff the results by plugin USB on/off of your device.

Set up your system to detect your device

The fix is mentioned on the setup page I mentioned previously.

Create a file called something like /etc/udev/rules.d/99-android.rules that contains rules telling udev to make the device node world-writable when a matching device is found.

sudo vi /etc/udev/rules.d/99-android.rules

SUBSYSTEMS=="usb", ATTRS{idVendor}=="2717", ATTRS{idProduct}=="9039", MODE="0666", OWNER="allex"

For simplicity, you could specify “*” for ATTRS so that all devices can be used

SUBSYSTEMS=="usb", ATTRS{idVendor}=="*", ATTRS{idProduct}=="*", MODE="0666", OWNER="your_name"

Manually update vendor ID to ~/.android/adb_usb.ini

If adb still cannot detect your device. the solution is that the ADB daemon needs also your USB device vendor ID (VID)

mkdir ~/.android/
echo 0x2717 > ~/.android/adb_usb.ini # 其中 0x2717 是通过 lsusb 得到的 VendorId

For more VID please see also at adbusb.ini and the Linux USB listing at http://www.linux-usb.org/usb.ids

Restart udev and adb services

sudo restart udev
adb kill-server && adb start-server
    
adb devices
List of devices attached 
66c95e6 device

DONE!

Enjoy android develops with adb logcat

resin 处理 jsp, html 页面乱码问题

## Solution 1:resin安装目录下conf/app-default.xml 文件

<servlet servlet-name="resin-file" servlet-class="com.caucho.servlets.FileServlet">
   <init>
      <character-encoding>utf-8</character-encoding>
   </init>
</servlet>

PS: resin是通过 com.caucho.servlets.FileServlet 来处理静态文件的.

## Solution 2:Put the following in your `web.xml` to achieve the goal.

<jsp-config>
   <jsp-property-group>
      <url-pattern>*.jsp</url-pattern>
      <page-encoding>UTF-8</page-encoding>
   </jsp-property-group>
</jsp-config>

See also:
Unicode – How to get the characters right? – JSP/Servlet response

## Solution 3:Setting character sets in JSPs.

You use the contentType attribute of the page directive to define character sets in JSPs, as the following example shows:

<%@ page contentType="text/html; charset=utf-8" pageEncoding="utf-8" %>

To use multiple character sets on a single page, use the UTF-8 character set encoding, as the following example shows:

<%@ page contentType="text/html; charset=utf-8" %>