latitude longitude - assign values to a big matrix in R -



latitude longitude - assign values to a big matrix in R -

i have matrix object, named location, 3 columns(id, latitude, logitude) , 18,289 rows:

# id latitude longitude # 320503 31.29530 120.5735 # 310104 31.18852 121.4365 # 310115 31.22152 121.5444 # 110105 39.92147 116.4431

i want calculate haversine distance between each id. in bundle geosphere function distm() can create symmetric distance matrix, dimension 18289 18289. however, r reported error cannot allocate vector of size 2.5 gb.

similarly, ff package,

data.distance.ff <- ff(0, dim = c(18289, 18289))

produces no errors, when assign values ff matrix, r produces errors come again.

data.distance.ff[1:18289, 1:18289] <- distm(location[, 2:3]

error: cannot allocate vector of size 2.5 gb in addition: warning messages: 1: in matrix(0, ncol = n, nrow = n) : reached total allocation of 2047mb: see help(memory.size) 2: in matrix(0, ncol = n, nrow = n) : reached total allocation of 2047mb: see help(memory.size) 3: in matrix(0, ncol = n, nrow = n) : reached total allocation of 2047mb: see help(memory.size) 4: in matrix(0, ncol = n, nrow = n) : reached total allocation of 2047mb: see help(memory.size)

i can verify error with:

data.distance.ff[1:10000, 1:10000] <- distm(location[1:10000, 2:3]

and error:

error: cannot allocate vector of size 772.1 mb.

my questions are:

is code assigning values ff matrix object wrong? should using special assign values ff object instead? canff objects handle storage requirement? can utilize method calculate distance using apply function not involve loops? know function distm() produces matrix twice big needed because symmetric. are there other methods handling big data? bigmemory bundle not appear work on windows computer.

r latitude-longitude apply ff

Comments

Post a Comment

Popular posts from this blog

Delphi change the assembly code of a running process -

Delphi change the assembly code of a running process - i'm trying alter address 00741fa5 has push test.009e721c . alter push test.009e71c8 . procedure callback; asm force $9e71c8 end; procedure tform2.btn1click(sender: tobject); var ppid :dword; pprocess : cardinal; begin getwindowthreadprocessid(findwindow(nil,pchar('test')), @ppid); pprocess := openprocess(process_all_access, false, ppid); injectasm(pprocess, $741fa5, 10, integer(@callback), 0); end; here injectasm function (with error checking) function injectasm(process: longword; injectaddress, injectsize, codeaddress, codesize : integer): pointer; var csize, rsize : integer; replaced : array of byte; jmp : array [0..4] of byte; jmpaddress : integer; nopv : byte; : integer; nbr: ulong_ptr; begin //injectsize must equal or greater 5, because need space our //(far) jmp wwxxyyzz instruction //if there's no space, inject in place of few instructions if injectsize ...
Read more

json - Hibernate and Jackson (java.lang.IllegalStateException: Cannot call sendError() after the response has been committed) -

json - Hibernate and Jackson (java.lang.IllegalStateException: Cannot call sendError() after the response has been committed) - hello have problem hibernate , jackson. have 2 pojos @entity @table(name = "users") public class user implements serializable { static final long serialversionuid = 4235637833262722256l; @id @generatedvalue private int id; @column(name = "creation_date") private date creationdate; @column(name = "first_name") private string firstname; @column(name = "last_name") private string lastname; @version @column(name = "version") private int version; @column(name = "e_mail") private string email; @column(name = "enabled") private boolean enabled; @column(name = "login") private string login; @column(name = "password") private string password; @jsonmanagedreference("role") @onetomany(mappedby = "user", cascade = cascadetype.all,fetch = fetchtyp...
Read more

C++ 11 "class" keyword -

C++ 11 "class" keyword - i've started using c++ 11 little bit more , have few questions on special uses of class keyword. know it's used declare class, there's 2 instances have seen don't understand: method<class t>(); and class class_name *var; why precede typename keyword class in first example, , keyword pointer in sec example? that known elaborated type specifier , necessary when class name "shadowed" or "hidden" , need explicit. class t { }; // love of god don't t t; t t2; if compiler smart, give these warnings: main.cpp:15:5: error: must utilize 'class' tag refer type 't' in scope t t2; ^ class main.cpp:14:7: note: class 't' hidden non-type declaration of 't' here t t; ^ if class not defined, deed forwards declaration. for example: template <typename t> void method() { using type = typename t::value_type; } method...
Read more