Appending Items to List in R

A very common programming problem is that a function must return a list of objects (itmes like trees...).

In R, we can do something like the following:

func_????? <- function ( ... )

{

    ret_TreeList = list()

    for(i in 1: ?????)

    {

        if(...)

        {

             ret_TreeList[[i]] = something

        }

        else(...)

        {

             ret_TreeList[[i]] = something else

        } 

        

    }

    return(ret_TreeList)

}

Then we cna use it like this:

>TreeList = func_?????(...)

>TreeList[3]

Hope it helps ;-}

Install R packages Removed from CRAN (from the source)

I need to use some really useful R packages but unfortunately they are removed from CRAN which means I cannot just install.packages(...) from the cloud and I must install it from the source.

There are two ways of doing this:

(1)

library(devtools)
install_url('http://cran.r-project.org/src/contrib/Archive/dynamo/dynamo_0.1.3.tar.gz')
install_url('http://cran.r-project.org/src/contrib/Archive/gafit/gafit_0.4.tar.gz')

(2) Which I prefer!!
Download the package source first and then:

install.packages('/path/dynamo_0.1.3.tar.gz', type = 'source')

ref:
http://stackoverflow.com/questions/18655976/how-to-install-the-package-that-has-been-removed-from-the-cran-repository-in-r-e

Where is to find the bodyfat dataset?

When trying some old (even not so old) examples using R package mboost, the dataset "bodyfat" cannot be loaded.

> data("bodyfat", package = "mboost")
Warning message:

In data("bodyfat", package = "mboost") : data set ‘bodyfat’ not found

Many famous examples are still like this, like
Yanchang Zhao's book: “R and Data Mining: Examples and Case Studies”

==============================================================

The data is provided in package TH.data, but no longer in package mboost.

To use the data, run code below.

data("bodyfat", package="TH.data")

Find the k Smallest Elements in a List using R

This program is definitely not the most efficient way to find the k smallest elements in a List. But I simply need this now. Improvements might be done later and you are mostly welcome to share any ideas of improvements.

l = c(7,3,6,1,0,2,10,9)

KSmallestElem <- function(DataList, k)

{

  n = length(DataList)

  kmin = {}

  

  initMinIndex = which.min(DataList)

  initMinVal = min(DataList)

  kmin = rbind(kmin,c(initMinIndex,initMinVal))

  

  # 1st level loop:

  #     k times to find the smallest k elemenst

  for(i in 1:k)

  {

    minIndex = 1

    minValue = DataList[1]

    # 2nd level loop:

    #     over the original DataList

    for(j in 1:n)

    {

      if(DataList[j] > kmin[i,2] &&

         DataList[j] < minValue)

      {

        minIndex = j

        minValue = DataList[j]

      }

      

    }

    

    kmin = rbind(kmin,c(minIndex,minValue))

  }

  

  return(kmin)

}

KSmallestElem(l,3)

#################################

Results:

> KSmallestElem(l,3)

     [,1] [,2]

[1,]    5    0

[2,]    4    1

[3,]    6    2

[4,]    2    3

> 

Reverse scaling data/ Unscaling data in R

In machine learning, data should usually be scaled before feeding to the training model since variables might be in different kinds of ranges. If not scaled, it shall make the prediction far from accuracy when calculating the distances among  data.

However, we still need a prediction result in the original range, which is the reverse of scaling.

In r, we could do this:

attributes(d$s.x)

d$s.x * attr(d$s.x, 'scaled:scale') + attr(d$s.x, 'scaled:center')

for example:

> x <- 1:10
> s.x <- scale(x)
> s.x
            [,1]
 [1,] -1.4863011
 [2,] -1.1560120
 [3,] -0.8257228
 [4,] -0.4954337
 [5,] -0.1651446
 [6,]  0.1651446
 [7,]  0.4954337
 [8,]  0.8257228
 [9,]  1.1560120
[10,]  1.4863011
attr(,"scaled:center")
[1] 5.5
attr(,"scaled:scale")
[1] 3.02765
> s.x * attr(s.x, 'scaled:scale') + attr(s.x, 'scaled:center')
      [,1]
 [1,]    1
 [2,]    2
 [3,]    3
 [4,]    4
 [5,]    5
 [6,]    6
 [7,]    7
 [8,]    8
 [9,]    9
[10,]   10
attr(,"scaled:center")
[1] 5.5
attr(,"scaled:scale")
[1] 3.02765

ref:
http://stackoverflow.com/questions/10287545/backtransform-scale-for-plotting

If it helps, you could also try the R library:
http://www.inside-r.org/packages/cran/DMwR/docs/unscale

Error in readRDS(file) : error reading from the connection

I met this error when I was installing the package "stringr". It is one of the dependencies if you want to install "ggplot2" package. I have this problem probably because my computer was interrupted when I was installing "ggplot2" package. But that's just a guess.

Right after the error, it prompts : "TRY removing /usr/local/lib/R/site-library/00LOCK-stringi/". So I just sudo went to that directory and rm -rf 00LOCK-stringi.

I solved the problem this way. But there are a popular ref online:
        http://stackoverflow.com/questions/6473831/readrdsfile-in-r

Try it if my solution doesn't work for you,

Deal with Missing Data NA in R

Firstly, Yes, there is no " " on NA.

I wrote something like

                   if(mat[i,j] == "NA"){...}

Please don't laugh at me R users...

I got my answer from the following post:
 http://stackoverflow.com/questions/27018471/r-missing-value-where-true-false-needed

comments = c("no","yes",NA)
for (l in 1:length(comments)) {
    if (!is.na(comments[l])) print(comments[l])
}
[1] "no"
[1] "yes"

So the function is.na is the way to do the judgement. And yes use NA directly, no need to quote.

R problem of installing packages: package ‘xxxx’ is not available (for R version 3.2.2) and unable to access index for repository xxxx

I met this problem after re-installed the Ubuntu. I've got to set up the whole programming environment again...  :-{{

--- Please select a CRAN mirror for use in this session ---
Warning: unable to access index for repository http://cran.utstat.utoronto.ca/src/contrib
Warning message:
package ‘xxxx’ is not available (for R version 3.2.2) 

Checked something online. The following two helped...

http://stackoverflow.com/questions/25721884/how-should-i-deal-with-package-xxx-is-not-available-for-r-version-x-y-z-wa
http://stackoverflow.com/questions/25599943/unable-to-install-packages-in-latest-version-of-rstudio-and-r-version-3-1-1

Basically, just change to another mirror...I changed to one in US, didn't work either... Then Canada(BC), WORKed!!!!

Wired...but you can try the same cuz there might be many problems to the unavailability of a university server, which is usually the case for R mirrors...

Happy coding!

VIM color scheme

With "syntax on" command in configuration file "~/.vimrc", the color scheme is really unacceptable for a programmer...

There are many beautiful alternatives. Some commonly used color scheme conf files can be find in Ubuntu directary: "/usr/share/vim/vim7x/colors/" such as desert.vim and morning.vim.

We only need to, in ~/.vimrc, add:

                  colo desert 

                  syntax on

...much better now!

But if you would like to use other color conf downloaded from somewhere else, you have to put the "xxxx.vim" file under the above directary so that vi could locate it automatically.

Hope it helps!

Unbreakable Linux, Windows and Me: Ubuntu Builder - Installation on Ubuntu 12.04LTS

Unbreakable Linux, Windows and Me: Ubuntu Builder - Installation on Ubuntu 12.04LTS: Ubuntu Builder is a very handy and powerful tool in building a custom Ubuntu linux distro. The following steps are taken for me to succes...

Installing .deb Packages on Ubuntu and Remove it

Sometimes it is more direct to install softwares on Ubuntu using .deb packages like in Windows using setup.exe. Here are some how-to commands.

(1) Installing:

                    sudo dpkg -i xxxxxxxxxxxx.deb

(2) Find out the list of all installed packages:

                    dpkg --list

                    dpkg --list | grep -i keywords

(3) Uninstalling:

With the pack name found using the commands above, we can remove the install.

                   sudo dpkg --remove xxxxxxxx

Continuing and please comment to add more...

WiFi Certificate File Install

  Some WiFi connections require a special certificate in Ubuntu, for example: ca-bundle.crt. And here is the way to install this kind of certs in Ubuntu.

1. copy the cert file to directory: /etc/ssl/certs/
    This is the place in Ubuntu to put the cert file and in other Linux distributions it might be a little different.

2. check the permissions on the cert file.








 Sometimes the original file is with the permission as the following when it has just been downloaded:

-rw------- 1 root root 660107 Oct 21 06:43 ca-bundle.crt

However, the cert file cannot been seen for users when selecting the cert file in filling up the WiFi information. So we need to change it to:

                      -rw-r--r-- 1 root root 660107 Oct 21 06:43 ca-bundle.crt

by command:
                      chmod 644 cert-cert

It means that "I can change it and everyone else can read it".

Now back to the WiFi security table and try to find the cert file, it can be seen.

Hope it helps and happy ubunt'n!

vi getting multiple “Sorry, the command is not available in this version…”

Just re-installed Ubuntu, but I am getting the following error codes when launching vi...I am using the configuration from:
    http://jsdelfino.blogspot.ca/2010/11/my-vi-configuration-file.html

Error detected while processing /home/boris/.vimrc:
line   30:
E518: Unknown option: autochdir
line   39:
E319: Sorry, the command is not available in this version: syntax on
line   43:
E319: Sorry, the command is not available in this version: filetype indent on
line   64:
E319: Sorry, the command is not available in this version: function! RESTORE_CURSOR()
line   68:
E319: Sorry, the command is not available in this version: endfunction
line   69:
E319: Sorry, the command is not available in this version: autocmd BufReadPost * call RESTORE_CURSOR()
Press ENTER or type command to continue

Soved by using suggestions from this bbs:

http://askubuntu.com/questions/284957/vi-getting-multiple-sorry-the-command-is-not-available-in-this-version-af

Try from within vim ...

:version

and if your get ...

Small version without GUI.

You are missing package vim-gui-common. It is probably also advisable to install vim-runtime. Commands to install:

sudo apt-get install vim-gui-common
sudo apt-get install vim-runtime

---

Another cause might be that alternatives is pointing to the wrong one:

update-alternatives --display vim

to show what is used and ...

update-alternatives --config vim

to change to another vim. It could be that /usr/bin/vim.gnome is used and you need /usr/bin/vim

---

########################################################

It worked for me right after I installed vim-gui-common:

sudo apt-get install vim-gui-common

Hope this helps!

It's because your R is too old

I just want to install ggplot2 for R. But it says I don't have reshape2. Then I tried to install reshape2 but it says
" plyr package for R not available for R version 3.0.2"
I thought this is because the R version is too new. Somebody said and I tried download the plyr package and install. FINALLY I GET:
"ERROR: this R is version 3.0.2, package 'plyr' requires R >= 3.1.0"
So, My R version is too old!!!!

I searched a lot. Only the following worked.

       http://askubuntu.com/questions/218708/installing-latest-version-of-r-base

       http://stackoverflow.com/questions/16093331/how-to-install-r-version-3

=======================================================================

This is an old post and I wrote it when it was impossible to install R with simple (it was installing old 2.15 version):

sudo apt-get install r-base-core

Right now you do not need to do all written below and can simply use the abovementioned command. At the time of updating it will give you 3.1 version.

Here is previous post

Uninstall old R

sudo apt-get remove r-base-core

Open sources.list

sudo nano /etc/apt/sources.list    

and add deb to it

deb http://cran.rstudio.com/bin/linux/ubuntu precise/

precise is your ubuntu name (may be different)

"precise" is 12.04. My ubuntu is 14.04. It should be "trusty"

Add key to sign CRAN packages

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9

Add specific PPA to the system

sudo add-apt-repository ppa:marutter/rdev
sudo apt-get update
sudo apt-get upgrade

installing

sudo apt-get install r-base

From R to check version

version

should give you

R version 3.2.2 (2015-08-14)

Hope people will find this helpful

Install Dropbox in Ubuntu

  Dropbox is my working directory but it doesn't work when it is just installed on Ubuntu. I did it in Software Center and it seemed to be installed. When I tried to launch it, it requires the password for super user, and then nothing happened.

I tried the following commands and it works.

          sudo rm -rf /var/lib/dropbox/.dropbox-dist

          dropbox start -i

  It can be seen that the Dropbox starts to downloading now. That is probably the actual installation and that's why the password is required before.


######################################################################

If still does not work for you...try to remove the Dropbox and re-install it again.

           sudo apt-get remove nautilus-dropbox

But there may be problems like:

  "dpkg status database is locked by another process"

Try
            sudo dpkg --configure -a
but not necessary it does the job...it just finishes the unfinished work...anyway it worked for me. Some even say dpkg should never be manually setup. I think it might be easier and safer to just reboot.

Now we could re-install the Dropbox from the .deb package.

            sudo dpkg -i dropbox_2015.02.12_amd64.deb 

Hope it works for you. If not, please comment to share the problems or solutions. Thanks in advance.

Installing chrome gives an error: “dependency is not satisfiable”

Once ubuntu is installed, there are a few routines needed.

http://howtoubuntu.org/things-to-do-after-installing-ubuntu-14-04-trusty-tahr#partners

When trying to install Chrome in Software Center, there might be error showing "dependency is not satisfieable"...

I solved the problem by:

sudo apt-get update && sudo apt-get install -f




This is from:

http://askubuntu.com/questions/46645/installing-chrome-gives-an-error-dependency-is-not-satisfiable

Python Data Structure -- defaultdict()

When using the original Python data structure -- dict(), there will be exception for KeyError if the key doesn't exist when accessing by d[key]. But with difaultdict(), only with a default factory function, it will be used as the default value of the key if the key does not exist. 

# -*- coding: utf-8 -*-
from collections import defaultdict

members = [
    # Age, name
    ['male', 'John'],
    ['male', 'Jack'],
    ['female', 'Lily'],
    ['male', 'Pony'],
    ['female', 'Lucy'],
]

result = defaultdict(list)
for sex, name in members:
    result[sex].append(name)

print result

# Result:
defaultdict(<type 'list'>, {'male': ['John', 'Jack', 'Pony'], 'female': ['Lily', 'Lucy']})

Ludovico Einaudi - Nuvole Bianche (Alexander Flemming)

  My piano, my old friend, has just been shipped to our new home. I would like to share this vedio to express my happiness that could not be described by words. There are many many great music piecies I would like to play during the past...7 or 8 years, but this one came to my mind first, alone with all the memories.

  I would like to start this new page "music" to record my music memories because it was, is, and yet to be the most loyal friend in my life. Some memory pieces are beyond words and they can only be remembered by music.

Use Your iPhone in Different Countries

When I tried to use my old iPhone 4 bought from China in other countries for example Canada, it works fine with calling and texting since it is unlocked. But the strange thing is the 3G network is not accessible.

The solution to this problem is to set up the APN profile. There are some apps you could find in the Appstore but some of them are not free and some need a login. But I found the following way which is easy and free.

Just open the link down below in safari and it will let you set your carrier, for example, Canada/Fido.

                          http://www.unlockit.co.nz/

Then it will jump to setting->general->profiles and just press install and you are done! You can now see a app with APN on it. And if you try to open it, it is still a webpage.

But when you change to another carrier company, you must delete the profile from setting->general->profiles (now the APN app icon should be gone) and repeate the process above again.

Hope it helps.

Some useful commands about SSH under Linux

If you are travelling constantly like me, then remote login would be a daily routine. So here are some commands you might find useful when working with SSH.

1. Login

Suppose the username on the server is 'boris', then open a terminal and put:

                 ssh boris@xxx.ee.university.com

That is username@host_domain.

Then a password is required and now you are in!

2. Download files from the Server to the local computer

If your current teminal are connected to the server now, start a new one without ssh connection. To download to the local computer, the working session must be locally. Then put:

       scp boris@xxx.ee.university.com:/home/employees/boris/... /home/myLocalMachine/work/...

That is scp theSameWithLogin:ServerFilePath LocalFilePath

To  be continued......

Visualize Data Density on 1 Dimension Axis

This source code clip is for visulizing data density on 1 axis like the following:

# reference:
# http://stackoverflow.com/questions/7352220/how-to-plot-1-d-data-at-given-y-value-with-pylab

import numpy as np
import matplotlib.pyplot as pp
import random

val = 0. # this is the value where you want the data to appear on the y-axis.
ar = np.random.random(size=40)
pp.plot(ar, np.zeros_like(ar) + val, '+')
pp.show()

To Start Up

I would like to start blogging today in English.

I used to keep a blog mostly in Chinese on http://blog.csdn.net/boriscoding. For this blog, I will start from translating some popular useful ones into English. And from now on, I will just blog in English on both blogs.

This blog, as my old one on CSDN, is my technique experience about coding and tinkering interesting new things. Sometimes it might look really easy and simple. That is because not everything in use has been learned in school. I must learn new technologies just by doing. And the point of this blog is to remind me that I did this before and the old solution might still be useful now. More importantly, hopefully, the blogs are here to help those who strive searching for the proper answers to the same problems I met.

Thank you in advance for valuable advices!