问题描述:

I'm calculating the frequency of words into many text files (140 docs), the end of my work is to create a csv file where I can order the frequence of every word by single doc and by all docs.

Let say I have:

absolut_freq= {u'hello':0.001, u'world':0.002, u'baby':0.005}

doc_1= {u'hello':0.8, u'world':0.9, u'baby':0.7}

doc_2= {u'hello':0.2, u'world':0.3, u'baby':0.6}

...

doc_140={u'hello':0.1, u'world':0.5, u'baby':0.9}

So, what I need is a cvs file to export in excel that looks like this:

WORD, ABS_FREQ, DOC_1_FREQ, DOC_2_FREQ, ..., DOC_140_FREQ

hello, 0.001 0.8 0.2 0.1

world, 0.002 0.9 0.03 0.5

baby, 0.005 0.7 0.6 0.9

How can I do It with python?

网友答案:

You can make it a mostly a data-driven process—given only the variable names of all the dictionary variables—by first creating a table with all the data listed in it, and then using the csv module to write a transposed (columns for rows swapped) version it to the output file.

import csv

absolut_freq = {u'hello': 0.001, u'world': 0.002, u'baby': 0.005}
doc_1 = {u'hello': 0.8, u'world': 0.9, u'baby': 0.7}
doc_2 = {u'hello': 0.2, u'world': 0.3, u'baby': 0.6}
doc_140 ={u'hello': 0.1, u'world': 0.5, u'baby': 0.9}

dic_names = ('absolut_freq', 'doc_1', 'doc_2', 'doc_140')  # dict variable names

namespace = globals()
words = namespace[dic_names[0]].keys()  # assume dicts all contain the same words
table = [['WORD'] + list(words)]  # header row (becomes first column of output)

for dic_name in dic_names:  # add values from each dictionary given its name
    table.append([dic_name.upper()+'_FREQ'] + list(namespace[dic_name].values()))

# Use open('merged_dicts.csv', 'wb') for Python 2.
with open('merged_dicts.csv', 'w', newline='') as csvfile:
    csv.writer(csvfile).writerows(zip(*table))

print('done')

CSV file produced:

WORD,ABSOLUT_FREQ_FREQ,DOC_1_FREQ,DOC_2_FREQ,DOC_140_FREQ
world,0.002,0.9,0.3,0.5
baby,0.005,0.7,0.6,0.9
hello,0.001,0.8,0.2,0.1
网友答案:

You could also convert it to a Pandas Dataframe and save it as a csv file or continue analysis in a clean format.

absolut_freq= {u'hello':0.001, u'world':0.002, u'baby':0.005}
doc_1= {u'hello':0.8, u'world':0.9, u'baby':0.7}
doc_2= {u'hello':0.2, u'world':0.3, u'baby':0.6}
doc_140={u'hello':0.1, u'world':0.5, u'baby':0.9}


all = [absolut_freq, doc_1, doc_2, doc_140]

# if you have a bunch of docs, you could use enumerate and then format the colname as you iterate over and create the dataframe
colnames = ['AbsoluteFreq', 'Doc1', 'Doc2', 'Doc140']


import pandas as pd

masterdf = pd.DataFrame()

for i in all:
    df = pd.DataFrame([i]).T
    masterdf = pd.concat([masterdf, df], axis=1)

# assign the column names
masterdf.columns = colnames

# get a glimpse of what the data frame looks like
masterdf.head()

# save to csv 
masterdf.to_csv('docmatrix.csv', index=True)

# and to sort the dataframe by frequency
masterdf.sort(['AbsoluteFreq'])
网友答案:

No matter how you want to write this data, first you need an ordered data structure, for example a 2D list:

docs = []
docs.append( {u'hello':0.001, u'world':0.002, u'baby':0.005} )
docs.append( {u'hello':0.8, u'world':0.9, u'baby':0.7} )
docs.append( {u'hello':0.2, u'world':0.3, u'baby':0.6} )
docs.append( {u'hello':0.1, u'world':0.5, u'baby':0.9} )
words = docs[0].keys()
result = [ [word] + [ doc[word] for doc in docs ] for word in words ]

then you can use the built-in csv module: https://docs.python.org/2/library/csv.html

相关阅读:
Top