Wavelet coherence website: http://www.pol.ac.uk/home/research/waveletcoherence/
Guide to wavelet analysis: http://paos.colorado.edu/research/wavelets/
Thursday, December 3, 2009
Wednesday, November 4, 2009
Research Fiasco
Today, in the midst of organizing my data and images to send to Dr. Dai, I accidentally erased over half of my research. I still have the raw data, and the Fortran code to produce it, but I lost my images, sampled data, sampling scripts and gnuplot scripts for making the images possible. That rm utility really removes the shit out of things. I think I can redo all the scripts in three hours.
%%%%%
Scripts in three hours? Perhaps not so much. I wrote a better Ruby script this time for organizing the Fortran data. Its still not beautiful, or even homely, but its better.
%%%%%
Ok, its been about six hours since I thought I could do this in three hours and I've (hopefully) finished writing my Ruby scripts for organizing the Fortran data... now to tackle the gnuplot scripts. Not so frequently asked questions is a great resource for gnuplotery. This is a good site for adding LaTeX magic to your gnuplots.
%%%%%
I just finished tweaking the gnuplot script. It is now 1:32 AM. Fantastic. Just in case something happens again, I'm going to paste my codes here. (I'm kind of embarassed. I know this code is ugly, I know that it can be prettified... but I can't spend time right now making it pretty/readable.)
Ruby:
#!/usr/bin/env ruby
#
# This script takes the .dat files output by the Fortran codes and organizes them into
# several figure .dat files to be turned into figures by the figures.plt gnuplot script.
#
# To run: ruby combine-data.rb
#
def process i1E, i2E, i3E, i1L, i2L, i3L
r = /(\d*\.\d+)(\s+)(\d+\.\d+)/
o1E = i1E.readlines; o1E.map!{|l| l.scan(r)}.flatten!(1); o1E.each{|l| l.delete_at(1)}
o2E = i2E.readlines; o2E.map!{|l| l.scan(r)}.flatten!(1); o2E.each{|l| l.delete_at(1)}
o3E = i3E.readlines; o3E.map!{|l| l.scan(r)}.flatten!(1); o3E.each{|l| l.delete_at(1)}
o1L = i1L.readlines; o1L.map!{|l| l.scan(r)}.flatten!(1); o1L.each{|l| l.delete_at(1)}
o2L = i2L.readlines; o2L.map!{|l| l.scan(r)}.flatten!(1); o2L.each{|l| l.delete_at(1)}
o3L = i3L.readlines; o3L.map!{|l| l.scan(r)}.flatten!(1); o3L.each{|l| l.delete_at(1)}
a = [o1E, o2E, o3E, o1L, o2L, o3L]
end
def print_data a, out1, out2, in1E, in2E, in3E, in1L, in2L, in3L
(0..380).each do |i|
if i%12 == 0 then
out1.print a[0][i][0]+" "+a[0][i][1]+" "+a[1][i][1]+" "+a[2][i][1]+"\n"
out2.print a[3][i][0]+" "+a[3][i][1]+" "+a[4][i][1]+" "+a[5][i][1]+"\n"
else
out1.print a[0][i][0]+" "+a[0][i][1]+"\n"
out2.print a[3][i][0]+" "+a[3][i][1]+"\n"
end
end
in1E.close; in2E.close; in3E.close
in1L.close; in2L.close; in3L.close
out1.close; out2.close
end
idata1E = File.open "ord1_m201_te_results.dat", "r"
idata2E = File.open "ord2_m51_te_results.dat", "r"
idata3E = File.open "ord4_m21_te_results.dat", "r"
idata1L = File.open "ord1_m201_tl_results.dat", "r"
idata2L = File.open "ord2_m51_tl_results.dat", "r"
idata3L = File.open "ord4_m21_tl_results.dat", "r"
out1 = File.open "fig-1.dat", "w"
out2 = File.open "fig-2.dat", "w"
a = process idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
print_data a, out1, out2, idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
idata1E = File.open "ord1_m201_te_dim_results.dat", "r"
idata2E = File.open "ord2_m51_te_dim_results.dat", "r"
idata3E = File.open "ord4_m21_te_dim_results.dat", "r"
idata1L = File.open "ord1_m201_tl_dim_results.dat", "r"
idata2L = File.open "ord2_m51_tl_dim_results.dat", "r"
idata3L = File.open "ord4_m21_tl_dim_results.dat", "r"
out1 = File.open "fig-3.dat", "w"
out2 = File.open "fig-4.dat", "w"
a = process idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
print_data a, out1, out2, idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
idata2E = File.open "ord2_te_peak_results.dat", "r"
idata2L = File.open "ord2_5ps_tl_results.dat", "r"
out1 = File.open "fig-5b.dat", "w"
out2 = File.open "fig-6b.dat", "w"
a = idata2E.readlines
a.each_with_index{|x,i| if i%6 == 0 then out1.print x+"\n" end }
a = idata2L.readlines
a.each_with_index{|x,i| if i%6 == 0 then out2.print x+"\n" end }
idata1E = File.open "ord2_m26_te_results.dat", "r"
idata2E = File.open "ord2_m51_te_results.dat", "r"
idata3E = File.open "ord2_m101_te_results.dat", "r"
idata1L = File.open "ord2_m26_tl_results.dat", "r"
idata2L = File.open "ord2_m51_tl_results.dat", "r"
idata3L = File.open "ord2_m101_tl_results.dat", "r"
out1 = File.open "fig-7.dat", "w"
out2 = File.open "fig-8.dat", "w"
a = process idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
print_data a, out1, out2, idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
idata1E = File.open "ord4_m21_te_results.dat", "r"
idata2E = File.open "ord4_m41_te_results.dat", "r"
idata3E = File.open "ord4_m51_te_results.dat", "r"
idata1L = File.open "ord4_m21_tl_results.dat", "r"
idata2L = File.open "ord4_m41_tl_results.dat", "r"
idata3L = File.open "ord4_m51_tl_results.dat", "r"
out1 = File.open "fig-9.dat", "w"
out2 = File.open "fig-10.dat", "w"
a = process idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
print_data a, out1, out2, idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
Gnuplot:
#
# This is a gnuplot script to produce the .ps and .tex files
# for use in LaTeX. It must be run in the same folder as the
# figure .dat files.
#
# To run: gnuplot figures.plt
#
set term pslatex auxfile
set style line 1 lt -1 lw 1
set style line 2 lt 3 pt 6 ps 2
set style line 3 lt 3 pt 10 ps 2
set output "fig-1.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{e}\;(K)$'
set xrange [0.1:2]; set yrange [300:1400]
plot "fig-1.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-1.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-1.dat" u 1:4 t "4th Order, M = 21" w p ls 3
set output "fig-2.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{l}\;(K)$'
set xrange [0.1:2]; set yrange [300:310]
plot "fig-2.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-2.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-2.dat" u 1:4 t "4th Order, M = 21" w p ls 3
set output "fig-3.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{e}/(\Delta T_{e})_{max}\;(K)$'
set xrange [0.1:2]; set yrange [0:1.1]
plot "fig-3.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-3.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-3.dat" u 1:4 t "4th Order, M = 21" w p ls 3
set output "fig-4.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{l}/(\Delta T_{l})_{max}\;(K)$'
set xrange [0.1:2]; set yrange [0:1.4]
plot "fig-4.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-4.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-4.dat" u 1:4 t "4th Order, M = 21" w p ls 3
set output "fig-5.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel '$r$, (mm)'; set ylabel '$T_{e}\;(K)$'
set xrange [0:0.0001]; set yrange [750:1400]
plot "fig-5a.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-5b.dat" u 1:2 t "2nd Order, M = 51" w p ls 2, \
"fig-5c.dat" u 1:2 t "4th Order, M = 21" w p ls 3
set output "fig-6.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel '$r$, (mm)'; set ylabel '$T_{l}\;(K)$'
set xrange [0:0.0001]; set yrange [306:310]
plot "fig-6a.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-6b.dat" u 1:2 t "2nd Order, M = 51" w p ls 2, \
"fig-6c.dat" u 1:2 t "4th Order, M = 21" w p ls 3
set output "fig-7.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{e}\;(K)$'
set xrange [0.1:2]; set yrange [300:1400]
plot "fig-7.dat" u 1:2 t "2nd Order, M = 101" w l ls 1, \
"fig-7.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-7.dat" u 1:4 t "2nd Order, M = 26" w p ls 3
set output "fig-8.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{l}\;(K)$'
set xrange [0.1:2]; set yrange [300:310]
plot "fig-8.dat" u 1:2 t "2nd Order, M = 101" w l ls 1, \
"fig-8.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-8.dat" u 1:4 t "2nd Order, M = 26" w p ls 3
set output "fig-9.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{e}\;(K)$'
set xrange [0.1:2]; set yrange [300:1400]
plot "fig-9.dat" u 1:2 t "4th Order, M = 51" w l ls 1, \
"fig-9.dat" u 1:3 t "4th Order, M = 41" w p ls 2, \
"fig-9.dat" u 1:4 t "4th Order, M = 21" w p ls 3
set output "fig-10.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{l}\;(K)$'
set xrange [0.1:2]; set yrange [300:310]
plot "fig-10.dat" u 1:2 t "4th Order, M = 51" w l ls 1, \
"fig-10.dat" u 1:3 t "4th Order, M = 41" w p ls 2, \
"fig-10.dat" u 1:4 t "4th Order, M = 21" w p ls 3
%%%%%
Like I said, its not pretty.
%%%%%
Scripts in three hours? Perhaps not so much. I wrote a better Ruby script this time for organizing the Fortran data. Its still not beautiful, or even homely, but its better.
%%%%%
Ok, its been about six hours since I thought I could do this in three hours and I've (hopefully) finished writing my Ruby scripts for organizing the Fortran data... now to tackle the gnuplot scripts. Not so frequently asked questions is a great resource for gnuplotery. This is a good site for adding LaTeX magic to your gnuplots.
%%%%%
I just finished tweaking the gnuplot script. It is now 1:32 AM. Fantastic. Just in case something happens again, I'm going to paste my codes here. (I'm kind of embarassed. I know this code is ugly, I know that it can be prettified... but I can't spend time right now making it pretty/readable.)
Ruby:
#!/usr/bin/env ruby
#
# This script takes the .dat files output by the Fortran codes and organizes them into
# several figure .dat files to be turned into figures by the figures.plt gnuplot script.
#
# To run: ruby combine-data.rb
#
def process i1E, i2E, i3E, i1L, i2L, i3L
r = /(\d*\.\d+)(\s+)(\d+\.\d+)/
o1E = i1E.readlines; o1E.map!{|l| l.scan(r)}.flatten!(1); o1E.each{|l| l.delete_at(1)}
o2E = i2E.readlines; o2E.map!{|l| l.scan(r)}.flatten!(1); o2E.each{|l| l.delete_at(1)}
o3E = i3E.readlines; o3E.map!{|l| l.scan(r)}.flatten!(1); o3E.each{|l| l.delete_at(1)}
o1L = i1L.readlines; o1L.map!{|l| l.scan(r)}.flatten!(1); o1L.each{|l| l.delete_at(1)}
o2L = i2L.readlines; o2L.map!{|l| l.scan(r)}.flatten!(1); o2L.each{|l| l.delete_at(1)}
o3L = i3L.readlines; o3L.map!{|l| l.scan(r)}.flatten!(1); o3L.each{|l| l.delete_at(1)}
a = [o1E, o2E, o3E, o1L, o2L, o3L]
end
def print_data a, out1, out2, in1E, in2E, in3E, in1L, in2L, in3L
(0..380).each do |i|
if i%12 == 0 then
out1.print a[0][i][0]+" "+a[0][i][1]+" "+a[1][i][1]+" "+a[2][i][1]+"\n"
out2.print a[3][i][0]+" "+a[3][i][1]+" "+a[4][i][1]+" "+a[5][i][1]+"\n"
else
out1.print a[0][i][0]+" "+a[0][i][1]+"\n"
out2.print a[3][i][0]+" "+a[3][i][1]+"\n"
end
end
in1E.close; in2E.close; in3E.close
in1L.close; in2L.close; in3L.close
out1.close; out2.close
end
idata1E = File.open "ord1_m201_te_results.dat", "r"
idata2E = File.open "ord2_m51_te_results.dat", "r"
idata3E = File.open "ord4_m21_te_results.dat", "r"
idata1L = File.open "ord1_m201_tl_results.dat", "r"
idata2L = File.open "ord2_m51_tl_results.dat", "r"
idata3L = File.open "ord4_m21_tl_results.dat", "r"
out1 = File.open "fig-1.dat", "w"
out2 = File.open "fig-2.dat", "w"
a = process idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
print_data a, out1, out2, idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
idata1E = File.open "ord1_m201_te_dim_results.dat", "r"
idata2E = File.open "ord2_m51_te_dim_results.dat", "r"
idata3E = File.open "ord4_m21_te_dim_results.dat", "r"
idata1L = File.open "ord1_m201_tl_dim_results.dat", "r"
idata2L = File.open "ord2_m51_tl_dim_results.dat", "r"
idata3L = File.open "ord4_m21_tl_dim_results.dat", "r"
out1 = File.open "fig-3.dat", "w"
out2 = File.open "fig-4.dat", "w"
a = process idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
print_data a, out1, out2, idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
idata2E = File.open "ord2_te_peak_results.dat", "r"
idata2L = File.open "ord2_5ps_tl_results.dat", "r"
out1 = File.open "fig-5b.dat", "w"
out2 = File.open "fig-6b.dat", "w"
a = idata2E.readlines
a.each_with_index{|x,i| if i%6 == 0 then out1.print x+"\n" end }
a = idata2L.readlines
a.each_with_index{|x,i| if i%6 == 0 then out2.print x+"\n" end }
idata1E = File.open "ord2_m26_te_results.dat", "r"
idata2E = File.open "ord2_m51_te_results.dat", "r"
idata3E = File.open "ord2_m101_te_results.dat", "r"
idata1L = File.open "ord2_m26_tl_results.dat", "r"
idata2L = File.open "ord2_m51_tl_results.dat", "r"
idata3L = File.open "ord2_m101_tl_results.dat", "r"
out1 = File.open "fig-7.dat", "w"
out2 = File.open "fig-8.dat", "w"
a = process idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
print_data a, out1, out2, idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
idata1E = File.open "ord4_m21_te_results.dat", "r"
idata2E = File.open "ord4_m41_te_results.dat", "r"
idata3E = File.open "ord4_m51_te_results.dat", "r"
idata1L = File.open "ord4_m21_tl_results.dat", "r"
idata2L = File.open "ord4_m41_tl_results.dat", "r"
idata3L = File.open "ord4_m51_tl_results.dat", "r"
out1 = File.open "fig-9.dat", "w"
out2 = File.open "fig-10.dat", "w"
a = process idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
print_data a, out1, out2, idata1E, idata2E, idata3E, idata1L, idata2L, idata3L
Gnuplot:
#
# This is a gnuplot script to produce the .ps and .tex files
# for use in LaTeX. It must be run in the same folder as the
# figure .dat files.
#
# To run: gnuplot figures.plt
#
set term pslatex auxfile
set style line 1 lt -1 lw 1
set style line 2 lt 3 pt 6 ps 2
set style line 3 lt 3 pt 10 ps 2
set output "fig-1.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{e}\;(K)$'
set xrange [0.1:2]; set yrange [300:1400]
plot "fig-1.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-1.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-1.dat" u 1:4 t "4th Order, M = 21" w p ls 3
set output "fig-2.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{l}\;(K)$'
set xrange [0.1:2]; set yrange [300:310]
plot "fig-2.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-2.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-2.dat" u 1:4 t "4th Order, M = 21" w p ls 3
set output "fig-3.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{e}/(\Delta T_{e})_{max}\;(K)$'
set xrange [0.1:2]; set yrange [0:1.1]
plot "fig-3.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-3.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-3.dat" u 1:4 t "4th Order, M = 21" w p ls 3
set output "fig-4.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{l}/(\Delta T_{l})_{max}\;(K)$'
set xrange [0.1:2]; set yrange [0:1.4]
plot "fig-4.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-4.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-4.dat" u 1:4 t "4th Order, M = 21" w p ls 3
set output "fig-5.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel '$r$, (mm)'; set ylabel '$T_{e}\;(K)$'
set xrange [0:0.0001]; set yrange [750:1400]
plot "fig-5a.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-5b.dat" u 1:2 t "2nd Order, M = 51" w p ls 2, \
"fig-5c.dat" u 1:2 t "4th Order, M = 21" w p ls 3
set output "fig-6.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel '$r$, (mm)'; set ylabel '$T_{l}\;(K)$'
set xrange [0:0.0001]; set yrange [306:310]
plot "fig-6a.dat" u 1:2 t "1st Order, M = 201" w l ls 1, \
"fig-6b.dat" u 1:2 t "2nd Order, M = 51" w p ls 2, \
"fig-6c.dat" u 1:2 t "4th Order, M = 21" w p ls 3
set output "fig-7.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{e}\;(K)$'
set xrange [0.1:2]; set yrange [300:1400]
plot "fig-7.dat" u 1:2 t "2nd Order, M = 101" w l ls 1, \
"fig-7.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-7.dat" u 1:4 t "2nd Order, M = 26" w p ls 3
set output "fig-8.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{l}\;(K)$'
set xrange [0.1:2]; set yrange [300:310]
plot "fig-8.dat" u 1:2 t "2nd Order, M = 101" w l ls 1, \
"fig-8.dat" u 1:3 t "2nd Order, M = 51" w p ls 2, \
"fig-8.dat" u 1:4 t "2nd Order, M = 26" w p ls 3
set output "fig-9.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{e}\;(K)$'
set xrange [0.1:2]; set yrange [300:1400]
plot "fig-9.dat" u 1:2 t "4th Order, M = 51" w l ls 1, \
"fig-9.dat" u 1:3 t "4th Order, M = 41" w p ls 2, \
"fig-9.dat" u 1:4 t "4th Order, M = 21" w p ls 3
set output "fig-10.tex"
set size 1,1
set xtics nomirror; set ytics nomirror
set format x "$%g$"; set format y "$%g$"
set xlabel 'Time, (ps)'; set ylabel '$T_{l}\;(K)$'
set xrange [0.1:2]; set yrange [300:310]
plot "fig-10.dat" u 1:2 t "4th Order, M = 51" w l ls 1, \
"fig-10.dat" u 1:3 t "4th Order, M = 41" w p ls 2, \
"fig-10.dat" u 1:4 t "4th Order, M = 21" w p ls 3
%%%%%
Like I said, its not pretty.
Tuesday, November 3, 2009
CSC-579 Image Classification Project
I'm going to cluster the features of each image and take the medioid of the clusters to be representative features. If need be, I'll take a few random outliers to complete the set. I've made a csv list of the cleaned keys, the keys of the images that had between 300 and 1200 keypoints. I'm going to take that list and break it up into 300 individual parts so that it can be clustered by Matlab. I might figure out some batch method in Weka and use that to cluster the features for each image.
%%%%%
Dude, omg. I found Ruby for Scientific Computing and I think it might save my balls on this project. There's a post about using jRuby and WEKA and I think I might go that route. I just unistalled WEKA. I was having trouble accessing it by `java -jar weka.jar`. I had originally downloaded via my synaptic package manager, and I think it installed it so that I could pull it up with just `weka`, but all the code snippets online use the java route. I don't know, I think this will work. I'm not really sure what's going on. Anyway, I'm going to have to figure out what jRuby is all about, but that doesn't look too hard.
%%%%%
Bingo. Downloading from the site made everything better. One day I'm going to be a Linux baller and just figure this stuff out in like a minute or something.
%%%%%
I have 300 images. Each image has 300 features. Each feature is a 132-tuple.
I can't handle all of this information at once. I can't reduce the number of images. Thus, I must reduce the number of features. I can do the following: (1) feature subset selection (3) discrete wavelet transform (2) principal component analysis. I might do some sort of tree grouping thing like DIANA or something, but I can't remember where that is in the book.
%%%%%
Attribute selection in Weka is in the "Filter" button. Here is a good site to start with.
%%%%%
I did the attribute selection on one of my arff files that contained the 300 keypoints for an image, and it returned the 7 best attributes - which corresponded to the columns. Basically, the 7 best dimensions for each keypoint, not the 7 best keypoints for the whole image. The solution: transpose the arff file so that the keypoints are the attributes listed (@attr... k1, k2, k3,...,k300), and the instances are the features of each keypoint.
%%%%%
I think I'm going to use Java to process the images for key features. This site has some Java/Weka codes and applets, and miscellaneous machine learning data sets. Here is a page from the Weka site concerned with using Jython with Weka. This page talks about calling the Weka API from Matlab. Apparently Matlab is built up from the JVM, I had no idea.
%%%%%
I've tried all day to run use Weka in an automated fashion. I'm going to try using it from Matlab, and then I'm going to just do it by... by hand. I had a the jruby.jar and weka.jar file in the directory, and some APIs just wouldn't work. I got some kind of exception like this:
Exception in thread "main" java.lang.NoClassDefFoundError: weka/filters/supervised/attribute/AttributeSelection
Caused by: java.lang.ClassNotFoundException: weka.filters.supervised.attribute.AttributeSelection
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: weka.filters.supervised.attribute.AttributeSelection. Program will exit.
I don't know what any of that means... it looks like it can't find the classes I need? I don't know. I ran the kmeans script from Scientifice Computing with Ruby, but I couldn't figure out how to adapt that to attribute selection, or even filtering.
I'm going to spend the rest of the night (a) writing a filtering script (b) mining all of my 300 classes by hand. Fortunately I have plenty of Diet Coke.
%%%%%
The other group got awesome results by averaging all of their descriptor vectors into one vector per image? Did I hear that right? What the fuck?
%%%%%
Okay, I found this site that has a bunch of Weka links including manuals and powerpoints. I think I finished scripting everything. I think that all we have to do now is figure out how to use weka effectively.
%%%%%
Here's a page that shows you how to use Weka in your Java code.
%%%%%
I was having a lot of trouble with exceptions in the Java code I used in the above website. Also, apparently in JRuby arrays come in two flavors: Ruby, and Java. I needed to pass a Java String[] array to a Weka API thingy, but I couldn't figure out how to declare a strictly Java array as opposed to a Ruby array. Eventually I found this wonderful site, the JRuby wiki. I don't know how I didn't find it earlier. Anyway, to Javafy something in JRuby, just do thing.to_java.
%%%%%
Dude, omg. I found Ruby for Scientific Computing and I think it might save my balls on this project. There's a post about using jRuby and WEKA and I think I might go that route. I just unistalled WEKA. I was having trouble accessing it by `java -jar weka.jar`. I had originally downloaded via my synaptic package manager, and I think it installed it so that I could pull it up with just `weka`, but all the code snippets online use the java route. I don't know, I think this will work. I'm not really sure what's going on. Anyway, I'm going to have to figure out what jRuby is all about, but that doesn't look too hard.
%%%%%
Bingo. Downloading from the site made everything better. One day I'm going to be a Linux baller and just figure this stuff out in like a minute or something.
%%%%%
I have 300 images. Each image has 300 features. Each feature is a 132-tuple.
I can't handle all of this information at once. I can't reduce the number of images. Thus, I must reduce the number of features. I can do the following: (1) feature subset selection (3) discrete wavelet transform (2) principal component analysis. I might do some sort of tree grouping thing like DIANA or something, but I can't remember where that is in the book.
%%%%%
Attribute selection in Weka is in the "Filter" button. Here is a good site to start with.
%%%%%
I did the attribute selection on one of my arff files that contained the 300 keypoints for an image, and it returned the 7 best attributes - which corresponded to the columns. Basically, the 7 best dimensions for each keypoint, not the 7 best keypoints for the whole image. The solution: transpose the arff file so that the keypoints are the attributes listed (@attr... k1, k2, k3,...,k300), and the instances are the features of each keypoint.
%%%%%
I think I'm going to use Java to process the images for key features. This site has some Java/Weka codes and applets, and miscellaneous machine learning data sets. Here is a page from the Weka site concerned with using Jython with Weka. This page talks about calling the Weka API from Matlab. Apparently Matlab is built up from the JVM, I had no idea.
%%%%%
I've tried all day to run use Weka in an automated fashion. I'm going to try using it from Matlab, and then I'm going to just do it by... by hand. I had a the jruby.jar and weka.jar file in the directory, and some APIs just wouldn't work. I got some kind of exception like this:
Exception in thread "main" java.lang.NoClassDefFoundError: weka/filters/supervised/attribute/AttributeSelection
Caused by: java.lang.ClassNotFoundException: weka.filters.supervised.attribute.AttributeSelection
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: weka.filters.supervised.attribute.AttributeSelection. Program will exit.
I don't know what any of that means... it looks like it can't find the classes I need? I don't know. I ran the kmeans script from Scientifice Computing with Ruby, but I couldn't figure out how to adapt that to attribute selection, or even filtering.
I'm going to spend the rest of the night (a) writing a filtering script (b) mining all of my 300 classes by hand. Fortunately I have plenty of Diet Coke.
%%%%%
The other group got awesome results by averaging all of their descriptor vectors into one vector per image? Did I hear that right? What the fuck?
%%%%%
Okay, I found this site that has a bunch of Weka links including manuals and powerpoints. I think I finished scripting everything. I think that all we have to do now is figure out how to use weka effectively.
%%%%%
Here's a page that shows you how to use Weka in your Java code.
%%%%%
I was having a lot of trouble with exceptions in the Java code I used in the above website. Also, apparently in JRuby arrays come in two flavors: Ruby, and Java. I needed to pass a Java String[] array to a Weka API thingy, but I couldn't figure out how to declare a strictly Java array as opposed to a Ruby array. Eventually I found this wonderful site, the JRuby wiki. I don't know how I didn't find it earlier. Anyway, to Javafy something in JRuby, just do thing.to_java.
Subscribe to:
Comments (Atom)