python - pandas dataframe format from elements with multiple identifiers -


i have csv file following format. imagine white spaces comma seperated.

                                    slot0                          slot1   serial     timestamp     height     width     score     height     width     score    .... fa125       2015_05      215.00     125.01    156.02    235.23     862.23    135.52   ....

this goes on thousands of rows , repeats many slot#'s pattern. slot# associated "height, width, , score" centered on. is, slot0 corresponds first height, width , score , slot1 corresponds second height, width , score. each slot has 3 measurements.

i'm having trouble finding best way stick data pandas.dataframe associate slot number particular heights, widths , scores, serial or timestamps.

one thing have thought of this, it's not clear if can better.

serial     timestamp  s0_height  s0_width  s0_score  s1_height  s1_width  s1_score  .... fa125       2015_05    215.00     125.01    156.02     235.23     862.23   135.52   ....

it seems little awkard in form, if it's way guess can manage.

# maybe this? pd.dataframe({'fsj1503007n-ct0': ['20150311_021738', 140, 123, 213]}, ['timestamp', 's0_height', 's0_score', 's0_width'])

keep in mind can adapt csv in way instantiate dataframe, problem i'm not sure best way create dataframe data.

thanks!

import pandas pd import numpy np # create string buffer, don't need if have csv file io import stringio    # replicate csv file structure line1 = ','.join(['slot' + str(x) x in range(3)]) + '\n' line2 = 'serial,timestamp,' + ','.join(['height', 'width', 'score']*3) + '\n' np.random.seed(0) data = np.random.randint(100, 1000, size=9) line3 =  'fa125,2015_5,'+','.join([str(x) x in data]) + '\n' csv_buffer = line1+line2+line3  out[40]: 'slot0,slot1,slot2\nserial,timestamp,height,width,score,height,width,score,height,width,score\nfa125,2015_5,784,659,729,292,935,863,807,459,109\n'   # read file, set first 2 columns index, rest using multi-level column index level1 = ['slot'+str(x) x in range(3)] level2 = ['height', 'width', 'score'] multi_level_columns = pd.multiindex.from_product([level1, level2])  df = pd.read_csv(stringio(csv_buffer), index_col=[0,1], skiprows=[0], header=0) df.columns = multi_level_columns  out[62]:                    slot0              slot1              slot2                              height width score height width score height width score serial timestamp                                                          fa125  2015_5       784   659   729    292   935   863    807   459   109  # can reshape original df  df.stack(level=0)  out[63]:                          height  score  width serial timestamp                             fa125  2015_5    slot0     784    729    659                  slot1     292    863    935                  slot2     807    109    459

-

Comments

Post a Comment

Popular posts from this blog

What Are The Best Universities In The World?

Harvard and Yale are often referred to as “the best of the best”. But when it comes to the highest ranked universities, these and other Ivy League schools don't even make it into the top five. So what are the world's best universities? Well, according to an index by the Times Higher Education, Stanford ranks 3rd in the world. Most people associate Stanford with tech-capital Silicon Valley, which is where the school is located. And in fact, it's largely because of Stanford that the region is now the global capital of entrepreneurship and technology. In the 1940's and 50's the university heavily encouraged students and faculty to start their own companies, eventually leading to companies like Google and Snapchat. And still today, the school is known to produce some of the most successful entrepreneurs. A Stanford degree is in extremely high demand, and the school maintains the lowest acceptance rate in the country, admitting less than 5 percent of applicants. Much ...
Read more

Hard vs. Soft Water: What's The Difference?

Hard or Soft? HOW AM I SUPPOSED TO KNOW? THEY BOTH LOOK EXACTLY THE SAME! Hey there, Sapna here for Dnews. From the outside, all water is seemingly the same: clear, mostly flavorless, two hydrogens, one oxygen. But once you get past the basics, water is actually pretty versatile. It can be distilled, filtered, tap, bottled, and if you're real fancy, there's water marketed as kosher, organic or kabbalah. But did you know water can also be hard or soft? No, this isn't to describe how the water actually feels. It actually has to do with its mineral content. Water becomes ‘hard' when it has a higher than normal concentration of minerals, primarily calcium and magnesium. Water picks up these minerals from rocks and soil as it flows through rivers, wells or other waterways. Soft water, on the other hand, has a low mineral content, either because it has not yet flowed through rocks or wells - like rain water - or because the rocks it has come in contact with have little to no...
Read more

java - Solr query version issue: Invalid version or the data in not in 'javabin' format -

as part of recent project, our solr search engine upgraded 5.0.0. program runs client works fine in production/qa, when run on local system, error message: caused by: java.lang.runtimeexception: invalid version or data in not in 'javabin' format @ org.apache.solr.common.util.javabincodec.unmarshal(javabincodec.java:99) this exception apache-solr-solrj-1.4.1.jar, would've thought needs replaced. when check (running) production code, same jar file there. however, java version on production server has been upgraded java version "1.7.0_79". on local system, i'm pointing @ 1.6 jre sts/eclipse. if change 1.7, start getting 51 vs. 50 class version error in different part of program, indicating there's issue converting 1.7, although again doesn't seem problem on production server. i have tested solr server running correctly via web client. returns data in json format. one question have is, mean when says "invalid version or data isn'...
Read more