Mundane Astrology and Data Mining (or Cycle Hunting)

For those who wish to pursue new theories based on the Old Wisdom and who wish to look more deeply into our vastly changing world from a different perspective.

Moderators: Noel Eastwood, James Strom

Post Reply
User avatar
dubbhism
Posts: 7
Joined: Mon Aug 08, 2016 10:55 pm
Contact:

Mundane Astrology and Data Mining (or Cycle Hunting)

Post by dubbhism » Sun Aug 28, 2016 9:55 pm

Hi everybody,

Nowadays we're in the age of Big Data, and this is not going to stop anytime soon. A lot of data (World Bank reports, weather, economy etc. etc.) is freely available for anyone who's interested. Data mining-related techniques are used by banks to predict your financial stability, or by Facebook to annoy you with 'personalized content' but what about mundane astrology? Personally, i expect that certain types of data may also be of interest to mundane astrologers. But i don't know of anyone actively using this kind of approach right now.

Anyone?

My own starting point would be Theodor Landscheidt (my favorite astrologer) who has analyzed cyclical events on earth (economic, seismic etc.) and related them to the activity of the sun. According to Landscheidt, and a growing number of mainstream sun-astronomers, sunspot activity is modulated significantly by the big planets. Landscheidt did a lot of his research in the 70's and 80's. I suspect there was a lot of hard manual labour involved. I also suspect that nowadays a combination of (free) open data, (free) analytical software and a bit of 'harmonic analysis' in the sense of Fourier Analysis (FFT) should also be able to detect patterns/cycles/etc. that are related to the cycles of the (big) planets, and who knows what else.

Right now i'm just thinking out loud about a general approach to this kind of research. Ideally, it should be useful to the astrological community as a whole (at least to the statistical types). A very general first hypothesis would be something like: the principles of astrology suggest that there should be Open Data Sets that have cyclic structures hidden inside them, which, if properly analyzed, line up significantly with the cycles of certain planets.

If we can get a feeling for the kind of data that works (why not try Landscheidt's articles for inspiration), we can ask more refined questions and measure other kinds of variables. Personally, i would start with a Fourier analysis (FTT) of interesting Open Data sets, and the game would be to find cyclic events that have a strong correlation with the cosmic cycles we're interested in. To play the game fair, it would be important to report all results of the analysis, not just the 'hits'. If a correlation is found, the best procedure would be to find a similar data set and test/verify the result.

1. Available data. These are some good starting points
UN Data http://data.un.org/
World Bank Open Data http://data.worldbank.org/
KDnuggets http://www.kdnuggets.com/datasets/index.html
US Government Open Data https://www.data.gov/
UK Open Data https://data.gov.uk/data/search
Time Series Data Library (Australia) https://datamarket.com/data/list/?q=provider:tsdl
Gapminder https://www.gapminder.org/data/
Open Data Handbook http://opendatahandbook.org/

2. Tools for Data mining
Rapidminer (free, has Fourier analysis on board) https://rapidminer.com/
Excel (has Fourier analysis on board)
Matlab (advanced data torture)

3. Examples of (Bad) Data Mining
http://developers-club.com/posts/144618/
http://www.informationisbeautiful.net/2011/horoscoped/
https://medium.com/@samim/a-i-astrology ... .7837l6wpe

4. Astrology and data mining - thoughts
http://lifesboringletstastedeath.blogsp ... ology.html
http://www.creativitymaximized.com/2016 ... vable.html

5. Educational
Sun Spot Analysis using Fourier/FFT http://serc.carleton.edu/introgeo/teach ... spots.html
Astrostatistics and Data Mining (free ebook) http://link.springer.com/book/10.1007%2 ... 614-3323-1
Landscheidt's legacy http://bourabai.kz/landscheidt/publications.htm
More Landscheidt https://landscheidt.wordpress.com/

Mersenne
Moderator
Posts: 97
Joined: Tue Nov 01, 2011 8:11 pm
Contact:

Re: Mundane Astrology and Data Mining (or Cycle Hunting)

Post by Mersenne » Mon Aug 29, 2016 5:47 pm

Hi dubbhism,

Yes, that time has indeed come when it will be impossible to hide the cycles.

I had begun trawling Google Trends- not exactly "big data", but very specific searches can be carried out using the program, and there is an immediate plot. For instance, you can easily see seasonal trends in "wristwatches" as one approaches Christmas, but some periodicities are harder to explain.

Right now I'm tied up with another project, but I'd certainly like to hear more of this.

Regards,

Mersenne.
See on this Forum:
Mersenne’s Astrological Statistics & Datasets
Mersenne’s Microcosm
Mersenne’s Transneptunians

User avatar
dubbhism
Posts: 7
Joined: Mon Aug 08, 2016 10:55 pm
Contact:

Re: Mundane Astrology and Data Mining (or Cycle Hunting)

Post by dubbhism » Wed Aug 31, 2016 1:24 am

Hey Mersenne,

Google itself is large enough... i'd call that Big Data as well ;-)

At the moment, i'm really just exploring and trying my luck. See what methods might work. Fourier Analysis is an accurate method, which i'm a little familiar with through music, so that's a first idea. But the art of statistics has many methods for many different types of data analysis. The good thing is that there are also lots of forums with college students who need to brush up their statistics, and many people are quite willing to help you with specific statistical problems (i will probably be in 'astronomer' disguise in such places ;-))

And then there's the data collecting-side of it. So-called 'time series' are interesting (lists of data, timed at regular intervals) but many 'open data' time series have a yearly frequency, which is probably a bit too crude for most purposes. So it's also about finding the right 'time series' with the right interval (daily, weekly, monthly, perhaps yearly) on the right subject. In that sense, i guess you'd have to get lucky every now and then, and bump into the right piece of data.

One approach might be to look at global data, but local data can be just as interesting. Vancouver crime data anyone? Vehicle theft, offence against a person, etc. from 2003 - 2008 with randomized address/neighborhood but monthly frequency. If crime was my thing i'd be checking this one out http://data.vancouver.ca/datacatalogue/crime-data.htm

User avatar
dubbhism
Posts: 7
Joined: Mon Aug 08, 2016 10:55 pm
Contact:

Re: Mundane Astrology and Data Mining (or Cycle Hunting)

Post by dubbhism » Wed Oct 05, 2016 1:53 am

Here's a little idea for a general approach to machine learning for astrology, after experimenting a bit with various kinds of data.

When data scientists use machine learning tools, the basic idea is that they create a mathematical model that can predict certain things after it has been 'trained' with a (large) set of data. Step two is to check and validate this model, using a second set of data, to see if the actual predictions of the model are correct or not. This way you can get averagely performing models, bad models, good models, and the idea is that most models can be improved by all kinds of tweaks. So how can this aproach be useful in astrology?

If you create a certain type of model and evaluate it, the performance might be 80%, meaning that the model predicts correctly 80% of the time, or something like that. Now here's the idea: suppose you again train the same model, with the same data, except for one thing: you add some series of astrological data as well (angles, cycles, other numeric stuff). Now compare the score to the score without the astrological data and see what happens. If the score improves significantly... you may have caught something.

This approach should be possible using for example neural networks. The free open source Rapidminer software has several neural network modules, including a state of the art 'deep learning' module.

So now you know how i'll be wasting my time over the next couple of weeks ;-)

Meanwhile, i have not found a good use for Fouries analysis yet, but i do like this so-called 'STL-decomposition' trick a lot (not available in Rapidminer but easy to do in another open source free software ecosystem for statistics called R) https://www.otexts.org/fpp/6/5

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest