Comments on: GHAPACK: A Library for the Generalized Hebbian Algorithm http://kurtgrandis.com/blog/2009/02/08/ghapack-a-library-for-the-generalized-hebbian-algorithm/ Software Engineering & Entrepreneurship Fri, 09 Jul 2010 06:40:00 -0400 http://wordpress.org/?v=2.9.2 hourly 1 By: Bob Carpenter http://kurtgrandis.com/blog/2009/02/08/ghapack-a-library-for-the-generalized-hebbian-algorithm/comment-page-1/#comment-10 Bob Carpenter Fri, 14 Aug 2009 00:50:46 +0000 http://kurtgrandis.com/blog/?p=31#comment-10 Stochastic algorithms, like Gorrell's GHA approach to SVD, are much faster at finding solutions quickly to a few decimal places of accuracy. With fixed floating-point arithmetic, it's all approximate, so it's just a matter of how much accuracy you need from your SVD. Just tweak the convergence parameters. I discuss this in <a href="http://lingpipe-blog.com/2009/04/08/convergence-relative-sgd-pegasos-liblinear-svmlight-svmper/" rel="nofollow">a blog entry</a> on SVMs and logistic regression, but the ideas also apply to SVD. I reimplemented the GHA approach as part of LingPipe. There's an <a href="http://alias-i.com/lingpipe/demos/tutorial/svd/read-me.html" rel="nofollow">SVD Tutorial</a> which goes over some of the classic demos. Stochastic algorithms, like Gorrell’s GHA approach to SVD, are much faster at finding solutions quickly to a few decimal places of accuracy. With fixed floating-point arithmetic, it’s all approximate, so it’s just a matter of how much accuracy you need from your SVD. Just tweak the convergence parameters.

I discuss this in a blog entry on SVMs and logistic regression, but the ideas also apply to SVD.

I reimplemented the GHA approach as part of LingPipe. There’s an SVD Tutorial which goes over some of the classic demos.

]]>