Strange problem of Singapore ICA’s SAVE system

2011/06/28

I was trying to apply for visa yesterday. After I clicked the “Proceed to submit” button, I always get a blank page. Cleared cookie/cache, still same problem. I thought it’s because too many people applying visa. I tried after mid-night, still same problem. I tried school’s computer today, also same. Tried Firefox, failed at even an earlier step.

Tried my laptop and it succeed. I figured out that the reason of failure is probably adding the site in trusted sites. The web system requires turning off popup blocker for http://www.psi.gov.sg and http://www.enets.com.sg. I did that, and in addition I added both sites as trusted site. This unnecessary step turned out to cause the failure. I guess the reason is probably that when http://www.psi.gov.sg is trusted while singpass site is not, they are in different security zones, and there is problem in their handshake.

I’m a bit curious on why Firefox fails, so I checked the error console. The reason turned out to be the use of location.href(newurl), which Firefox considers as a property, not a method. If I manually types the url into the address bar, I can at least get to the singpass login page, which is one step further than the IE’s problem.

How to steal iPhone ringtone from iTunes shop?

2010/04/09

In iTunes shop, all musics and ringtones have 30 seconds preview. Ringtones are always less than 30 seconds. That means the ringtone previews are always in full length. If we can hear it, we can download it (for free, of course). This article is to share on how to download ringtones and add to your iPhone for free. It works for all ringtones in iTune Store. The main intention is to use this as an example to illustrate common practices in network hacking.

First, get a network log of iPhone’s ringtone preview traffic. To do this, setup a sniff-able wifi environment. This might be difficult for some people, but I have an existing environment. All my internet traffic goes through my linux router, so I simply run tcpdump there. I use wireshark to analyze the saved dump file.

When I play a ringtone preview, I see the this request: http://a1778.phobos.apple.com/us/r1000/031/Music/9c/22/14/mzi.wphahwgb.aac.p.m4p. (Lucky it’s not https. If it’s https, I have to try self signed certificate and see if it can pass the check.) Quickly do a direct wget. I get error 403 forbidden. First reaction is user agent. Change UA to iPhone. succeed.

$ wget -U 'Apple iPhone OS v3.1.3 CoreMedia v1.0.0.7E18' 'http://a1778.phobos.apple.com/us/r1000/031/Music/9c/22/14/mzi.wphahwgb.aac.p.m4p'
--2010-04-09 01:53:43-- http://a1778.phobos.apple.com/us/r1000/031/Music/9c/22/14/mzi.wphahwgb.aac.p.m4p
Resolving a1778.phobos.apple.com... 124.155.222.67, 124.155.222.58
Connecting to a1778.phobos.apple.com|124.155.222.67|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 493781 (482K) [text/plain]
Saving to: “mzi.wphahwgb.aac.p.m4p”

100%[=============================================================>] 493,781 256K/s in 1.9s

2010-04-09 01:53:45 (256 KB/s) - “mzi.wphahwgb.aac.p.m4p” saved [493781/493781]

Feed the m4p to a media player. It plays.

Up to here, we can already download the ringtone. Just follow previous steps and get the m4p url. It’s not very convenient though, as we have to use iphone and sniff to get the url. I want to get rid of the iphone step. I want to have a script to download a ringtone given it’s name, or have a script to download top 100 ringtone of a given genre.

For the top 100 script, I found out the url of top ringtone listing by genre to be something like http://ax.itunes.apple.com/WebObjects/MZStore.woa/wa/viewTop?selected-tab-index=0&top-ten-m=1%27%3B1&genreId=8004. However, the viewTop page requires sign-in. There are two ways to deal with sign-in. The hardworking way is to figure out the sign-in protocol and implement it. Usually it requires posting user id and password and get a session id. The dirty way is to sniff and get the session id, but we can only use the session before it expires. I’m not going into details about this. Here is the wget command to download the page. It’s a bit long because of those special X-apple-* headers. You can’t use it because 1. the session has expired; and 2. I have modified some of those IDs for my privacy. The page is an xml containing titles, artists, purchasing information, preview-url (the most important one for us), user ratings, etc.

wget -O - --header 'X-Apple-Store-Front: 143441-1,2' --header 'X-Apple-Partner: origin.0' --header 'X-Apple-Connection-Type: WiFi' --header 'X-Apple-Cuid: 068c5db16ca2b6956f7d582690613b68' --header 'X-Apple-Software-Cuid: 6a26ef98bfc6b1ef6f00694e61735a64' --header 'X-Dsid: 1369530585' --header 'X-Apple-Client-Application: WiFi-Music' --header 'Cookie: mz_at0=xQQUAABxlwAABABLsXbOCow79QEJcf6OqeR9C9ya+U87hxY=; mzf_in=180805; X-Dsid=1369530585; a=A2dAjgAAABtjAlRWMEsHtXFZZzAvdWlodAFxSTs5Ak1yOTjaSG9lYmtLdWcjKgsQAAdAJ5BiPTt=; Pod=18; s_cvp35b=%5B%5B%27google%253A%2520organic%27%2C%271369276708021%27%5D%2C%5B%27192.168.0.1%253A8000%27%2C%274278433477967%27%5D%5D; s_vi=[CS]v1|25C941528801054F-70001710E0178F3F[CE]; s_vnum_sg=ch%3Dip%26vn%3D1%3B; s_vnum_us=ch%3Dlegal%26vn%3D1%3Bch%3Dwebapps%26vn%3D3%3Bch%3Dip%26vn%3D2%3Bch%3Ddeveloper%26vn%3D1%3B' -U 'iTunes-iPhone/3.1.3 (2)' 'http://ax.itunes.apple.com/WebObjects/MZStore.woa/wa/viewTop?selected-tab-index=0&top-ten-m=1%27%3B1&genreId=8004'

There are many articles teaching how to add ringtones to iphone. I briefly describe here.

  1. Rename to .m4r
  2. Import (drag) to iTunes. It should appear under the ringtone directory in iTunes. Note: Don’t manually manage music and ringtone. Add to iTunes and sync. I tried the first way and failed miserably. I hate iTunes.
  3. You may want to change the metadata. Alternatively, before importing to iTunes, you can use opensource tools like mp4tags from libmp4v2 to change metadata. I prefer mp4tags, because it works in command line so that I can run in batch.
  4. Sync
  5. You should see the new ringtone in your iPhone.

So is it possible for apple to prevent this? I can think of a few solutions, but none of them work well.

  1. Do not provide preview. Customers won’t be happy.
  2. Add noise to preview. Shorten it to 10 seconds. Customers won’t be so happy.
  3. Use https or a custom protocol. “If we can hear it, we can download it.” It only makes hackers taking longer time. But, hey, hackers are the group of people having least money and most time.

Convert videos from iPhone using ffmpeg on Fedora 11

2010/03/11

ffmpeg in fedora 11 doesn’t buildin faac library, which encodes AAC. I need to build my own ffmpeg from source.

  1. yum install lame-devel xvidcore-devel x264-devel faad2-devel faac-devel gsm-devel dirac-devel libogg-devel libtheora-devel speex-devel libvorbis-devel openjpeg-devel liboil-devel schroedinger-devel libraw1394-devel libdc1394-devel bzip2-devel alsa-lib-devel xorg-x11-proto-devel libXau-devel libxcb-devel libXdmcp-devel libX11-devel libvdpau-devel libXext-devel libXv-devel libXvMC-devel
    Some packages are in rpmfusion. You know what you need to do.
  2. download ffmpeg source and extract. I downloaded the latest version 0.5.1.
  3. ./configure --arch=pentium4 --enable-bzlib --enable-libdc1394 --enable-libdirac --enable-libfaad --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-vdpau --enable-x11grab --enable-avfilter --enable-avfilter-lavf --enable-postproc --enable-swscale --enable-pthreads --enable-gpl --disable-stripping --cpu=pentium4 --enable-nonfree --enable-libfaac --prefix=/home/atp/install/ffmpeg-0.5.1
    I followed the configuration of ffmpeg from rpmfusion. The only changes made are:

    • --enable-nonfree --enable-libfaac
    • --prefix=/home/atp/install/ffmpeg-0.5.1(I never install my build using root.)
    • change i586 to pentium4 and removed some gcc options I don’t understand.
    • remove --disable-mmx2 --disable-sse --disable-ssse3 --disable-yasm
    • change to static build
  4. make
    make install

to be continued…

Noise problem with iTunes optimization

2010/03/10

I noticed severe image noise after I transferred my 320×480 photos to my iPhone. This is probably to do with the so called “optimization” done by iTunes.

Below is my original image:

Original Image

Original Image

Below is the “processed” image: (How did I get it? Select the image in iPhone and send email.)

Processed Image

Processed Image

Notice the added noise and slightly increased saturation. I tried to google to find out a way to disable the processing. No luck.

I tried to run process monitor on iTunes and found out the optimization is done by iTunesPhotoProcessor.exe. The processed image was saved into a .ithmb file. After a couple of hours, I couldn’t figure out a way to prevent the optimization.

Here is another attempt: Below is a comparison of the two JPEG header information:
ExifTool Version Number : 8.00
File Name : original.jpg
Directory : .
File Size : 41 kB
File Modification Date/Time : 2010:03:09 23:16:14+08:00
File Type : JPEG
MIME Type : image/jpeg
JFIF Version : 1.02
Resolution Unit : None
X Resolution : 100
Y Resolution : 100
Quality : 80%
DCT Encode Version : 100
APP14 Flags 0 : [14], Encoded with Blend=1 downsampling
APP14 Flags 1 : (none)
Color Transform : YCbCr
Image Width : 320
Image Height : 480
Encoding Process : Baseline DCT, Huffman coding
Bits Per Sample : 8
Color Components : 3
Y Cb Cr Sub Sampling : YCbCr4:4:4 (1 1)
Image Size : 320x480

ExifTool Version Number : 8.00
File Name : processed.jpg
Directory : .
File Size : 55 kB
File Modification Date/Time : 2010:03:09 08:25:10+08:00
File Type : JPEG
MIME Type : image/jpeg
JFIF Version : 1.01
Resolution Unit : None
X Resolution : 1
Y Resolution : 1
Image Width : 320
Image Height : 480
Encoding Process : Baseline DCT, Huffman coding
Bits Per Sample : 8
Color Components : 3
Y Cb Cr Sub Sampling : YCbCr4:2:0 (2 2)
Image Size : 320x480

The most suspicious differences are X&Y Resolution and YCbCr. Could any of these be the culprit? For example, I can generate an image with the same parameter as the processed image and hope iTunes will skip the processing. I haven’t tried this method yet…

China Map Deviation as a Regression Problem

2010/01/22

All published maps of China are deviated. GPS devices sold in China are modified to give the same deviated coordinates. If you don’t know, you may read here, here, here or here. Fortunately, the same deviation algorithm is applied on all maps I have seen, including Garmin (unistrong in China) GPS maps, mapabc.com, Google Maps/Earth. The algorithm is secrete and is only accessible by authority and companies such as garmin and google. Needless to say, this is very annoying for GPS users. Many individuals tried to discover the deviation algorithm by GPS measurement and correlation and found the algorithm to be not only nonlinear but very complicated to describe.

I accidentally found the Chinese version of Google Map ditu.google.com to be able to correlate satellite image with map, and it gives the amount of deviation for any location in China. This URL queries the deviation of 34.29273N,108.94695E (Xi’an): http://ditu.google.com/maps/vp?spn=0.001,0.001&t=h&z=18&vp=$34.29273,108.94695 (seems it’ doesn’t work now)

With enough sample data, we should be able to get a regression function, which, should resemble the deviation algorithm. I’m not good at regression so I’m putting up all my data and hope someone can help out. It can be downloaded from here. The format is very simple: four fields (longitude, latitude, longitude deviation and latitude deviation) separated by tab. Longitude deviation means (deviated_longitude – true_longitude). The points are sampled with 0.025 degree separation, i.e. 40 samples per degree. There are 1529737 points (lines of text) in the file. Only points in mainland China are available. Figure 1 and 2 shows an overview of the data.

There is another file, which contains samples from 8 selected lines (4 west-east, 4 south-north). The sample resolution is higher (200 samples per degree). It is used to plot Figure 3-6. I think it’s helpful for regression analysis.

Here are the plots of the data:

latitude deviation shown in color

Fig. 1. latitude deviation shown in color


longitude deviation shown in color

Fig. 2. longitude deviation shown in color


Fig. 3. longitude deviation v.s. longitude


Fig. 4. latitude deviation v.s. longitude


Fig. 5. longitude deviation v.s. latitude


Fig. 6. latitude deviation v.s. latitude

Some observations:

  1. The longitude deviation is always positive (deviate to the east). The maximum is 0.0085562 degree.
  2. The latitude deviation ranges from -0.0038542 (to the south) to +0.0028230 (to the north) degree.
  3. It’s very obvious that there are sinusoid component of period 1 and 1/3. (see Fig. 3, 4 and 6)
  4. Fig. 4. looks simple. You may think it’s f(x)=b*sin(a*x) + b*sin(3*a*x) + c*x + d. You are wrong. There are other small components.
  5. To make discussion easier, let’s define fdx(x,y) to be the longitude deviation of a point with longitude x and latitude y. Similarly, fdy(x,y) to be the latitude deviation of that point. So, Figure 3 shows fdx(x,25.12), fdx(x,32.24), … Figure 6 shows fdy(85.52,x), fdy(97.84,x)
  6. I suggest using fourier transform, but I’m not good at it.

Happy regression!

C Container Library

2009/10/05

A container library is a data structure library for containing data. Common examples are stack, hash-table, tree and queue. Container libraries for C++ and Java are standardized by the Standard Template Library and the Java Collections Framework. However, C programs such as the Linux kernel, GTK/GLib, Apache httpd, usually implement their own modules for individual projects. There are a few generic C container libraries which I will discuss later. For some reason, none of them don’t get much attention, not to mention getting standardized. Browsing through the latest Fedora and Ubuntu packages, I don’t find any C container related library. (If you know any, please let me know by leaving a comment here.)

Before discussing individual existing c container libraries, I will give way to categorize them by memory management.

  • user-managed
    User of the library manages the container’s data structure memory. Usually the container data structure is put together with the data. The most notable example is the Linux kernel linked list.

    struct student {
       int student_id;
       /* This is the container DS. */
       struct list_head list;
    };
    
    struct list_head *pos;
    list_for_each(pos, &head) {
       /* list_entry() is just pointer arithmetic */
       struct student *stu = list_entry(pos, struct student, list);
       printf("%d\n", stu->student_id);
    }
    

    The main advantage of this type is memory efficiency, because container DS struct list_head is allocated together with data DS struct student.

  • lib-managed
    TO BE CONTINUED…
  • immortal

Why I don’t like C++

2009/06/19

I love C and Java, but I don’t like C++. C++ gives you lots of new stuff on top of C, but programming language isn’t supermarket, the more the better. Programming language shouldn’t go ad-hoc or evolution. It should go intelligent design.

C++ gives classes, inheritances, information encapsulation … lots of nice OO stuff. But on the other hand, it allows pointer manipulation. WTF! It’s like establishing a comprehensive, wonderful law, but the last rule says “You can break all the previous rules.”

C allows passing parameters by value or pointer. C++ introduces pass-by-reference which is semantically the same as pass-by-pointer but syntactically different. Being able to do the same thing in a thousand ways is not a plus for programing languages, I’d vote it to be a minus.

The only feature of C++ I like is variable declaration between statements, especially “for (int i=0 …”.

Star Charts

2009/02/07

The 6 star charts were generated using pp3.

vernal equinox at equator at midnight
star chart during vernal equinox at equator at midnight

summer solstice at equator at midnight
star chart during summer solstice at equator at midnight

autumnal equinox at equator at midnight
star chart during autumnal equinox at equator at midnight

winter solstice at equator at midnight
star chart during winter solstice at equator at midnight

North Pole
star chart at North Pole

South Pole
star chart during South Pole

Hello world!

2009/01/23

Welcome to WordPress.com. This is your first post. Edit or delete it and start blogging!