FORWARD BASE B

"Pay my troops no mind; they're just on a fact-finding mission."

Category Archives: technology

Toward A Post-Western Aesthetic

At heart, the problem with Western Civilization is that it falls into the trap of the peacock’s tail striving for constant growth with flashy results but little thought of basic utility, sustainability, or resilience in the face of sudden shocks.

So great is the focus on competing for dominance now that no one has the time to think centuries ahead.

The truth is Western civilization and its philosophy was utterly discredited by the 20th century with its World Wars, Communist mega-states that killed off tens of millions, disasters like the Great Depression and Spanish Flu that the destruction and despair made possible.

Let’s face it, no one really believes in their hearts anymore that the future is good or meaningful and there’s no going back to the way it was.  The mood of the collective subconscious has been pre-apocalyptic since around the year 2000 and now among the most dominant symbols in the modern imagination are zombies that represent social alienation and atomization and evil clowns that represent mass sociopathy and chaos.

Symbol of a society where no one knows their neighbor and has to compete against everyone in a dehumanized rat race. Zombies originally a metaphor among Africans for how slavery took away one’s previous identity.

Symbol of a society where you have to adopt dark triad traits to survive. And our secret dissatisfaction with such a social order. Secretly many of us would just like to watch it all burn but can’t admit it even to ourselves.

The evolution of the West has produced impressive deer antlers but has created a society that is top-heavy and fragile.
The cost spiral started to get out of control with the rise of professional armies and the end of the feudal system until now it costs a trillion dollars to design a new weapon system, let alone manufacture, maintain, or deploy it and nearly a trillion every year to pay for old people and their medical care.  Soon, even all the wealth in the world can’t pay the basic bills and eventually even ever-growing loans can’t fill all the gaps.

Western civilization, with its stock markets, its winner-take-all economic systems and social status markets worships the growth pattern of short-lived weeds that choke the ground in a hurry before winter wipes them all away.  

I would rather worship the growth pattern of lichens that creep slowly, meticulously reinforcing each new growth area against hardship.
I would envision an ideal future civilization as being minimalistic, practical, and durable in its applications of technologies.
 
Extreme disparity in cost is why a single division worth of guerillas can challenge all the world’s great powers in the Middle East, why a guy with an ak-47 riding on a donkey beats a cruise missile or a drone, why a handful of men with boxcutters can shift the course of a nuclear superpower for decades, why an illegal immigrant Mexican roofer can have 3 kids while a salary striver that makes 70k a year can’t even afford 1 kid.
In future warfare, high technology will be limited to targets of appropriate value and low value forces will try to exert stategic pressure that forces cost-inefficient responses.

Much of the vanity of West is its stubborn insistence dating back to Christianity and reinforced by the Enlightenment that man inhabits a different universe than the rest of nature.  In truth the same laws that govern planets, rocks, animals, and bacteria bind humans just the same.

To set the tone for a society that works with the nature of the universe rather than try to defy it, we must begin to imagine what a Post-Western society might look and feel like.
In my next post I will begin to explore a possible aesthetic for such an order, according to my own sensibilities.  

My aesthetic will be austere, minimal, defensive, yet embrace a sort of stark beauty in contrast to a modern civilization of neon-bright advertisements that overload the eyes as high fructose corn syrup overwhelms the palate.

But it also won’t echo the old West cathedrals and row-houses with their straight lines and rigid rectangles, however pretty or pleasing they may be.
My aesthetic will try to appear more fluid, representing a lack of boundaries between man and nature, between intellect and the flesh.  For it’s precisely such dichotomies that gave rise to the West’s most debilitating neuroses.

Francis Bacon Predicted Airplanes, Submarines, and Lasers in the 17th Century

“Nylon and air-conditioning wouldn’t have surprised Sir Francis Bacon. He predicted them, along with most of our other present scientific wonders, over 300 years ago!…

this astounding man foresaw the airplane, television, movies, submarines, automobiles almost the whole range of modern discoveries.

Recently a research scientist, digging through the latin script of this ancient work checked off the list of these three hundred-year-old predictions and found that every one of them had come true except one!”

LINK

 

Many of these predictions arose in Bacon’s “New Atlantis”,  where he writes about an idealized society ruled by scientific method and where the rulers are scientists.

Not only did Bacon’s ideas lead to modern science and the enlightenment, but he also predicted something very like the modern secular state.

I find Bacon interesting because we see in our present world enlightenment ideas taken to an absurd extreme by ideological types.

Bacon was a shrewd politician and statesman who had a nuanced understanding of the real world.   He would never have believed in childish black and white extremes such as  “a war of all against all” or a “noble savage” that plague our civilization to the present day.

I imagine he would weep if he came back and saw where the intellectual movement he helped begin has ended up.

Francis Bacon perhaps gives us a glimpse of what the “enlightenment” could have been like had it been guided by worldly wisdom and savvy rather than reckless idealism and bookish wishful theory.

 

Philosophy was his hobby after his day job as a senior figure in Elizabeth I's government.

Philosophy was his hobby after his day job as a senior figure in Elizabeth I’s government.

 

The Dawn of an Age of Solar Flight?

From the 1950s to the 1990s, engineers explored the limits of what was possible in aviation.

In so doing, however, they hit inevitable barriers of cost effectiveness.

We’d made our point with elaborate manned space missions but had no financial incentives to keep blowing fortunes on them.

The concorde, a supersonic jet liner made by the Europeans proved so expensive that not even rich people could keep it commercially afloat.

Super expensive space shuttles with nearly 1/60 odds of blowing up on take off haven’t proved worth replacing.

The age of exorbitant spending just to prove a point is over.

First, new technologies and engineering projects must be cost effective.

And this brings up an obvious frontier in flight if we want to be as energy effective as possible.

Solar aircraft.

It would seem we’re approaching a point where batteries can store enough solar power to get through the night or in bad weather.

Just imagine the larger implications of this…

How long until people actually start living in the air to get away from the power of governments and rent seekers?

LINK

Swiss solar plane

How Microsoft’s Human Resources Culture Drove Away Talent

“Microsoft’s implementation – “stack ranking”, a bell curve that pits employees and groups against one another like rats in a cage – plunged the company into internecine fights, horse trading, and backstabbing.

…every unit was forced to declare a certain percentage of employees as top performers, then good performers, then average, then below average, then poor…For that reason, executives said, a lot of Microsoft superstars did everything they could to avoid working alongside other top-notch developers, out of fear that they would be hurt in the rankings.

Employees quickly realised that it was more important to focus on organisation politics than actual performance:…”

LINK

One of the commenters on this article, a ‘mikesmith’ gives some real food for thought by challenging current conventional wisdom about the ascendancy of STEM.

“I was particularly struck by the very last line, quoting Jobs: ‘Microsoft never had the humanities and liberal arts in its DNA.’ That is so insightful and so true, and says so much about why Apple is now the world’s biggest company. And it’s now fundamentally an arts company, nont a tech company. Sure it has to have great technology. But the purpose of the technology is to sell the arts products. It’s the products created by musicians, writers, filmmakers and others sold on iTunes that is financing Apple’s growth.

And it’s not just true about companies but about countries as well. Those that prosper in coming years will be those that promote the arts, the humanities, the liberal arts. Education in those areas should be a country’s number one priority, and those countries that do that will be the leaders and will have the most prosperous economies. People who study business and technology simply aren’t capable of coming up with the creative ideas. They are good at bean-counting, or finding ways of making the creative peoples’ ideas work better, but they shouldn’t be in charge and they certainly shouldn’t be receiving the bulk of the investment. It’s the artists and the creatives who matter, who now generate the ideas and the profits. Consider how much economic activity just two artists, Tolkien and Rowling, have generated in the past decade. Huge streams of billions, stemming from the work of just two artists! And they will continue for decades.

I’ve read numerous articles in this paper and others recently that young people are studying business and the sciences more than the arts and the humanities. That is just disastrous, both for the individuals and for their countries. And I feel so sorry for those young people who have been brainwashed into thinking that that’s the way to go. There’s no future for them. They should consider well Jobs’ insight, it’s quite brilliant.”

Nanotechnology in Ancient Times

“According to a 2000-year-old recipe for hair dye, the ancient Greeks and Romans were harnessing a scientific force that they had no idea even existed – they were using nanotechnology on their very own heads.
The Greeks and Romans used hair dye with some measure of frequency, most often for the purpose of dying their gray hair to black. Their dry mixture contained ingredients such as slaked lime and lead oxide, which – when exposed to human hair for approximately 3 days – causes nanocrystals made from lead sulfide to form inside the shaft of hair.
This reaction is caused when sulfur from the amino acids that are naturally present in hair keratins mix with the lead in lead oxide – initially, this is what causes the hair to turn black, but it apparently also causes lead sulfide nanocrystals that are highly similar to those found in modern, advanced scientific processes!
In simpler terms, the chemical compound that forms inside of the human hair is what colors the hair without damaging it – and the process by which the hair is dyed black is very similar to modern nanotechnology. Fortunately for the Greeks and Romans, this kind of lead-based hair dye is safe for human use, since the compound typically has trouble penetrating the skin.
Interestingly enough, the chemical engineering that came from this dye process – where the tiny crystal structures line up to form ‘quantum dots‘ – is something that scientists have admitted is a “current challenge in nanotechnology”, and is actually a process that researchers are currently trying to figure out how to develop on their own.”

LINK

quantum dots picture

These substances were colored by quantum dots in a modern laboratory. The principle at work is related to that used in ancient hair dye

The Big Science and Technology Problems of the 21st Century

The big problems are mostly the same as in the 20th century and most of them stretch back much farther than that.

http://www.networkworld.com/community/blog/what-are-grand-technology-and-scientific-challenges-21st-century

In fact, X Prize last year it declared a top eight list of key challenges that could end up being public competitions in the coming months or years.  The eight concepts or challenges included:

1. Water (“Super ‘Brita’ Water Prize”) – Develop a technology to solve the world’s number one cause of death: Lack of safe drinking water:

2. Personal Health Monitoring System (“OnStar for the Body Prize”) – Develop and demonstrate a system which continuously monitors an individual’s personal health-related data leading to early detection of disease or illness.

3. Energy & Water from Waste – Create and demonstrate a technology that generates off-grid water and energy for a small village derived from human and organic waste.

4. Around the World Ocean Survey – Create an autonomous underwater vehicle that can circumnavigate the world’s oceans, gathering data each step of the way.

5. Transforming Parentless Youth – Dramatically and positively change the outcome for significantly at risk foster children, reducing the number of incarcerations and unemployment rate by fifty-percent or more.

6. Brain-Computer Interface (“Mind over Matter”) – Enable high function, minimally invasive brain to computer interfaces that can turn thought into action.

7. Wireless Power Transmission – Wireless transmission of electricity over distances greater than 200 miles while losing less than two percent of the electricity during the transmission.

8. Ultra-Fast Point-To-Point Travel – Design and fly the world’s fastest point-to-point passenger travel system

#1 is probably done. Though it’s possible to create solutions at different scales of production.

#2 is going to be interesting as hackers will add functions to their sensors, and malicious ones will disrupt other peoples sensors for fun and profit.

I’ve heard of many implentations of #3, so it’s going to come down to what is most economical.

#4 is probably done, though a more robust version that can go deeper will be required to really satisfy the spirit of the goal.

#5 is quite difficult considering everything in our economy is forcing more people to be unemployed in the traditional sense. This is a judo problem, you can’t fix it within the normal means.

On #6, I’ve seen some simple EEG style sensors that can be integrated into games, but for the most part Brain-Machine interfaces are Sci-Fi. It’s easier to run prosthetics off of nerve impulses coming through limbs rather by sensing brainwaves without implants. So it’s going to take awhile to crack that problem. 3d interfaces are hitting the market now, both in VR headsets and 3d intractable  xbox kinect sensors:

The skeleton drawing system the kinect sensors use is software-based and can be modified, but other companies have already launched “improved” sensors that can be used on their own for 3d interaction.

#7 is interesting and we’ll have to see what is the most economical way of tackling it.

#8 needs to factor in safety, otherwise it won’t be widely used.

Some of the NRC’s problems are less thrilling, the benefits aren’t as clear to the man on the street, and it sort of reads like a list of “stuff we were going to do anyway, but we made a report for it”:

From the National Research Council report, the five challenges are:

1. How can the U.S. optics and photonics community invent technologies for the next factor of-100 cost-effective capacity increases in optical networks?

2. How can the U.S. optics and photonics community develop a seamless integration of photonics and electronics components as a mainstream platform for low-cost fabrication and packaging of systems on a chip for communications, sensing, medical, energy, and defense applications?

3. How can the U.S. military develop the required optical technologies to support platforms capable of wide-area surveillance, object identification and improved image resolution, high-bandwidth free-space communication, laser strike, and defense against missiles?

4. How can U.S. energy stakeholders achieve cost parity across the nation’s electric grid for solar power versus new fossil-fuel-powered electric plants by the year 2020?

5. How can the U.S. optics and photonics community develop optical sources and imaging tools to support an order of magnitude or more of increased resolution in manufacturing?

More interestingly, there is no way these questions can cover the whole of desires and needs that technology must fill for the 21st century. What are they missing?

Automated Hydroponics Garden With Arduino

Arduino is useful because it adds the ability to interact with the real world with technology in a very flexible way. If anyone thinks technology hasn’t brought practical gains to everyday life, they need to check it out.

To see what others have built with Arduino check out this page:

http://www.instructables.com/tag/?sort=none&q=arduino&limit:type:id=on

A few tutorial websites:

http://www.arduino.cc/en/Tutorial/HomePage

http://www.ladyada.net/learn/arduino/

http://www.arduinotutorials.com/

Hydroponics

This is the first of a series of articles on hydroponic growing tecniques, in this article I am going to introduce you to the advantages (and disadvantages) of hydroponic culture, this type of culture is very useful for those of you that want to grow pesticide-free, sustainable, nutritious fruits, vegetables and herbs indoor (closets, small rooms). Modern food is often polluted by chemicals, hormones and it is often deficient of many essential nutrients like vitamins and mineral salts. Growing plants and herbs that are rich of these essential nutrients is relatively cheap with hydroponics. You can even grow fresh fodder for cattle in order to produce high quality meat , cheese and milk.

My next articles will also cover more complex arguments like homemade fertilizers, greenhouse culture, aeroponics and how to build lighting and irrigation systems. There is quite a market for over-priced equipment and fertilizers so it is important to learn how to build your own in order to save a lot of $$$, this requires only basic skills.

For those of you that aren’t adept at basic horticulture I have included a series of links to books that explain basic information like: how plants grow, how photosynthesis works, why PH is important for plants etc You can find this links at the end of this post, I have also included links for some very important books about hydroponics.

INTRODUCTION

Hydroponic is a type of culture in which all nutrients are supplied to the plant through the irrigation water, with the growing substrate being soilless (mostly inorganic). In open systems the nutrient solution is discarded after it pass through root mass/medium while in closed systems the solution recirculate.

ADVANTAGES

Crops can be grown where no suitable soil exists or where the soil is contaminated with disease.

Labor for tilling, cultivating, fumigating, watering, and other traditional practices are largely eliminated.

Maximum yields are possible, making the system economically feasible in high-density and expensive land areas.

Conservation of water and nutrients, closed systems cause no form of pollution.

Complete control over the growing environment.

DISADVANTAGES

More expensive than traditional soil culture.

Require basic skills in various disciplines: electrical, plumbing, chemistry, horticulture.

The plants react very fast to poor nutritious or bad enviromental conditions.

Disease can spread very quickly if proper care is not given.

Not all plant cultivars are suitable for hydroponic culture.

THERE ARE 2 THINGS YOU NEED TO KNOW BEFORE STARTING

Number 1

This culture is very profitable for off-season high value crops, for example basil, cherry tomatoes, peppers and herbs during the winter. If you can sell your harvests to gourmet restaurants and farmer-markets you can recover your growing costs very quickly. You can grow different high quality crops in only 1 square meter using a shelf-like structure or rotary hydroponic systems.

Number 2

Lighting costs are very high:

Fluorescent tubes, High Intensity Discarge Lamps can be bought at a fair price BUT they are inefficient, consume a lot of electricity and most light bulbs/tubes don’t come in the right light spectrum therefor up to 60% of emitted light is wasted. They contain mercury and HID lamps generate a lot of excess heat; They also require expensive reflectors and ballasts for optimal use. It is important that you use these lamps only for herbs and leaf vegetables if you want to grow tomatoes or other crops that require a lot of light you must build a LED Array with blue and red leds (with 10% yellow/green light).

LUMENS ARE USELESS FOR PLANTS LIGHTING REQUIREMENTS, you need to use PAR WATTS / square meter or microeinsteins. Plants require mostly blue and red photons for photosynthesis, you only need 10% of yellow-green light, most commercial lights are in the yellow green spectrum.  There is a lot of bulls*it in horticulture marketing especially regarding grow lights sold at hydroponic shops, most of these lamps are industrial lamps with just a different label.

LINKS

For Beginners:

http://www.simplyhydroponic.com/hydroponic-systems/5-types-of-hydroponic-systems

Horticulture:

http://rapidshare.com/files/416333364/Horticulture.rar

http://depositfiles.com/files/dezn3ocd6

Hydroponics:

http://avaxhome.ws/ebooks/science_books/biology_genetics/65m745364n645.html

http://www.mediafire.com/?t1avo5u5p187ni6

Tomatoes:

http://ebookee.org/Tomatoes_1002408.html

Scientific Studies:

http://hortsci.ashspublications.org/

Open Source Intelligence Analysis – We NSA Now

Working Thoughts:

1. Wikileaks can act as a secondary database. What we’ve seen so far makes it clear that most of the classified material is common knowledge but it could be useful.
2. Robert Steele is right that the humanitarian goodwill approach is superior. We’ve spent a lot of money in Afghanistan, but most of it was spent in unpopulated areas that were safe, the people who needed it didn’t get it. Lots of corruption. A tighter approach could be made.
3. Fiverr and penpal sites can also be useful for general cultural understanding or simple local tasks, e.g. : http://fiverr.com/worryfustion/help-you-learn-about-the-ethnic-groups-in-vietnam

http://fiverr.com/vann97/answer-10-questions-in-great-details-about-vietnam
4. Nearly all current prediction markets operate as zero-sum or negative-sum markets.


More OSINT Links:

“Dradis is a self-contained web application that provides a centralised repository of information to keep track of what has been done so far, and what is still ahead.”

http://dradisframework.org/

Links for OSINT (Open Source Intelligence) by Randolph Hock
http://www.onstrat.com/osint/

City Data:
http://www.city-data.com/

Public Records:
http://publicrecords.onlinesearches.com/

Name/Location Search Engine:
https://pipl.com/

“creepy is an application that allows you to gather geolocation related information about users from social networking platforms and image hosting services. The information is presented in a map inside the application where all the retrieved data is shown accompanied with relevant information (i.e. what was posted from that specific location) to provide context to the presentation.”
http://ilektrojohn.github.com/creepy/

Here is a recent example that uses the Palantir platform and OSINT:

Less than four months ago, the Southern portion of Sudan seceded and formed South Sudan, only the 5th country to be created this century. In this session, we will demonstrate how Palantir can draw from a plethora of Open Source Intelligence (OSINT) data sources (including academic research, blogs, news media, NGO reports and United Nations studies) to rapidly construct an understanding of the conflict underlying this somewhat anomalous 21st Century event. Using a suite of Palantir Helpers developed for OSINT analysis, the video performs relational, temporal, statistical, geospatial, and social network analysis of over a dozen open sources of data.

See also:

Detecting Emergent Conflicts through Web Mining and Visualization

https://www.recordedfuture.com/assets/Detecting-Emergent-Conflicts-through-Web-Mining-and-Visualization.pdf

&

Maltego

http://www.paterva.com/web6/

Open Source Intelligence Analysis – Palantir Does Indeed Kick Ass

Messing around with the Palantir Government suite right now. You can get an account and mess around with it here:

https://analyzethe.us/

You have the ability to import/export data, filter access, set up collaborative teams and access to the open archives of the US Gov and some non profits. There are two tiers of users, novice users and power users:

Workspace Operations
Restrictions for Novice Users
Importing data

Novice users can only import data that is correctly mapped to the deployment ontology. Power users are exempt from this restriction.

The maximum number of rows in structured data sources that a Novice user can imported at one time is restricted by the NOVICE_IMPORT_STRUCTURED_MAX_ROWS system property. The default value for this property is 1000.

The maximum size of unstructured data sources that can be imported by a Novice user at one time is restricted by the NOVICE_IMPORT_UNSTRUCTURED_MAX_SIZE_IN_MB system property. The default value for this property is 5 megabytes.
Tagging text

The maximum number of tags that a Novice user can create using the Find and Tag helper is restricted by the system property NOVICE_FIND_AND_TAG_MAX_TAGS. The default setting for this property is 50.

Novice users cannot access the Tag All Occurrences in Tab option in the Browser’s Tag As dialog.
SearchAround search templates

Novice users cannot import SearchAround Templates from XML files.

Novice users cannot publish SearchAround templates for use by the entire deployment, and cannot edit published templates.
All other SearchAround features remain available.
Resolving Nexus Peering data conflicts
The Pending Changes application is available only in the Palantir Enterprise Platform, and is only accessible to Workspace users who belong to the Nexus Peering Data Managers user group.
Nexus Peering Data Managers use the Pending Changes application to check for, analyze, and resolve data conflicts that are not automatically resolved when a local nexus is synchronized with a peered nexus.
Deleting objects

Novice users cannot delete published objects.

Novice users cannot delete objects created or changed by other users.
Resolving objects

The maximum number of objects that Novice users can resolve together at one time is restricted by the NOVICE_RESOLVE_MAX_OBJECTS system property. This restriction does not apply to objects resolved by using existing object resolution suites in the Object Resolution Wizard or during data import.

Novice users may use the Object Resolution Wizard only when using existing object resolution suites. Novice users cannot perform Manual Object Resolution, and cannot record new resolution criteria as an Object Resolution Suite.
To learn more, see Resolving and Unresolving Objects in Workspace: Beyond the Basics.
Map application restrictions
All map metadata tools in the Layers helper are restricted.
Novice users cannot access features that allow sorting of layers by metadata, coloring by metadata, or the creation of new metadata. All other Layer helper functions remain available.

In case you didn’t get what I just said, you have access the same tools the FBI and CIA use, except some minor limitations and no access to classified documents. If you have access to Wolfram Alpha/Mathematica and can google for history on your topic of interest then most of the classified files will become redundant.

What about data mining on a budget?

Consider relying on a GPU(s). A CPU is designed to be multitasker that can quickly switch between actions, whereas a Graphical Processing Unit(GPU) is designed to do the same calculations repetitively while giving large increases in performance. The stacks in the listed papers, while giving exponentially higher speeds, did not use modern designs or graphics cards, which hindered them from running even faster.

http://www.azintablog.com/2010/10/16/gpu-large-scale-data-mining/

The GPU (Graphics Prossessing Unit) is changing the face of large scale data mining by significantly speeding up the processing of data mining algorithms. For example, using the K-Means clustering algorithm, the GPU-accelerated version was found to be 200x-400x faster than the popular benchmark program MimeBench running on a single core CPU, and 6x-12x faster than a highly optimised CPU-only version running on an 8 core CPU workstation.

These GPU-accelerated performance results also hold for large data sets. For example in 2009 data set with 1 billion 2-dimensional data points and 1,000 clusters, the GPU-accelerated K-Means algorithm took 26 minutes (using a GTX 280 GPU with 240 cores) whilst the CPU-only version running on a single-core CPU workstation, using MimeBench, took close to 6 days (see research paper “Clustering Billions of Data Points using GPUs” by Ren Wu, and Bin Zhang, HP Laboratories). Substantial additional speed-ups are expected were the tests conducted today on the latest Fermi GPUs with 480 cores and 1 TFLOPS performance.

Over the last two years hundreds of research papers have been published, all confirming the substantial improvement in data mining that the GPU delivers. I will identify a further 7 data mining algorithms where substantial GPU acceleration have been achieved in the hope that it will stimulate your interest to start using GPUs to accelerate your data mining projects:

Hidden Markov Models (HMM) have many data mining applications such as financial economics, computational biology, addressing the challenges of financial time series modelling (non-stationary and non-linearity), analysing network intrusion logs, etc. Using parallel HMM algorithms designed for the GPU, researchers (see cuHMM: a CUDA Implementation of Hidden Markov Model Training and Classification by Chaun Lin, May 2009) were able to achieve performance speedup of up to 800x on a GPU compared with the time taken on a single-core CPU workstation.

Sorting is a very important part of many data mining application. Last month Duane Merrill and Andrew Grinshaw (from University of Virginia) reported achieving a very fast implementation of the radix sorting method and was able to exceed 1G keys/sec average sort rate on an the GTX480 (NVidia Fermi GPU). Seehttp://goo.gl/wpra

Density-based Clustering is an important paradigm in clustering since typically it is noise and outlier robust and very good at searching for clusters of arbitrary shape in metric and vector spaces. Tests have shown that the GPU speed-up ranged from 3.5x for 30k points to almost 15x for 2 million data points. A guaranteed GPU speedup factor of at least 10x was obtained on data sets consisting of more than 250k points. (See “Density-based Clustering using Graphics Processors” by Christian Bohm et al).

Similarity Join is an important building block for similarity search and data mining algorithms. Researchers using a special algorithm called Index-supported similarity join for the GPU to outperform the CPU by a factor of 15.9x on 180 Mbytes of data (See “Index-supported Similarity Join on Graphics Processors” by Christian Bohm et al).

Bayesian Mixture Models has applications in many areas and of particular interest is the Bayesian analysis of structured massive multivariate mixtures with large data sets. Recent research work (see “Understanding the GPU Programming for Statistical Computation: Studies in Massively Massive Mixtures” by Marc Suchard et al.) has demonstrated that an old generation GPU (GeForce GTX285 with 240 cores) was able to achieve a 120x speed-up over a quad-core CPU version.

Support Vector Machines (SVM) has many diverse data mining uses including classification and regression analysis. Training SVM and using them for classification remains computationally intensive. The GPU version of a SVM algorithm was found to be 43x-104x faster than SVM CPU version for building classification models and 112x-212x faster over SVM CPU version for building regression models. See “GPU Accelerated Support Vector Machines for Mining High-Throughput Screening Data” by Quan Liao, Jibo Wang, et al.

Kernel Machines. Algorithms based on kernel methods play a central part in data mining including modern machine learning and non-parametric statistics. Central to these algorithms are a number of linear operations on matrices of kernel functions which take as arguments the training and testing data. Recent work (See “GPUML: Graphical processes for speeding up kernel machines” by Balaji Srinivasan et al. 2009) involves transforming these Kernel Machines into parallel kernel algorithms on a GPU and the following are two example where considerable speed-ups were achieved; (1) To estimate the densities of 10,000 data points on 10,000 samples. The CPU implementation took 16 seconds whilst the GPU implementation took 13ms which is a significant speed-up will in excess of 1,230x; (2) In a Gaussian process regression, for regression 8 dimensional data the GPU took 2 seconds to make predictions whist the CPU version took hours to make the same prediction which again is a significant speed-up over the CPU version.

If you want to use the GPUs but you do not want to get your hands “dirty” writing CUDA C/C++ code (or other languages bindings such as Python, Java, .NET, Fortran, Perl, or Lau) then consider using MATLAB Parallel Computing Toolbox. This is a powerful solution for those who know MATLAB. Alternatively R now has GPU plugins. A subsequent post will cover using MATLAB and R for GPU accelerated data mining.

These are space whales flying through the sun:

Open Source Intelligence Analysis – Software, Methods, Resources

http://www.kurzweilai.net/intelligence-agencies-turn-to-crowdsourcing

Research firm Applied Research Associates has just launched a website, Global Crowd Intelligence, that invites the public to sign up and try their hand at intelligence forecasting, BBC Future reports.

The website is part of an effort called Aggregative Contingent Estimation, sponsored by the Intelligence Advanced Research Projects Activity (Iarpa), to understand the potential benefits of crowdsourcing for predicting future events by making forecasting more like a game of spy versus spy.

The new website rewards players who successfully forecast future events by giving them privileged access to certain “missions,” and also allowing them to collect reputation points, which can then be used for online bragging rights. When contributors enter the new site, they start off as junior analysts, but eventually progress to higher levels, allowing them to work on privileged missions.

The idea of crowdsourcing geopolitical forecasting is increasing in popularity, and not just for spies.  Wikistrat, a private company touted as “the world’s first massively multiplayer online consultancy,” was founded in 2002, and is using crowdsourcing to generate scenarios about future geopolitical events. It recently released a report based on a crowdsourced simulation looking at China’s future naval powers.

Warnaar says that Wikistrat’s approach appears to rely on developing “what-if scenarios,” rather than attaching a probability to a specific event happening, which is the goal of the Iarpa project.

Paul Fernhout put together a good open letter awhile back on the need for this, it seems IARPA has put some effort forward for this purpose:

http://www.phibetaiota.net/2011/09/paul-fernhout-open-letter-to-the-intelligence-advanced-programs-research-agency-iarpa/

A first step towards that could be for IARPA to support better free software tools for “crowdsourced” public intelligence work involving using a social semantic desktop for sensemaking about open source data and building related open public action plans from that data to make local communities healthier, happier, more intrinsically secure, and also more mutually secure. Secure, healthy, prosperous, and happy local (and virtual) communities then can form together a secure, healthy, prosperous, and happy nation and planet in a non-ironic way. Details on that idea are publicly posted by me here in the form of a Proposal Abstract to the IARPA Incisive Analysis solicitation: “Social Semantic Desktop for Sensemaking on Threats and Opportunities”

So what kind of tools can an amateur use for making sense of data?

Data Mining and ACH

Here is a basic implementation of ACH:

http://competinghypotheses.org/

Analysis of Competing Hypotheses (ACH) is a simple model for how to think about a complex problem when the available information is incomplete or ambiguous, as typically happens in intelligence analysis. The software downloadable here takes an analyst through a process for making a well-reasoned, analytical judgment. It is particularly useful for issues that require careful weighing of alternative explanations of what has happened, is happening, or is likely to happen in the future. It helps the analyst overcome, or at least minimize, some of the cognitive limitations that make prescient intelligence analysis so difficult. ACH is grounded in basic insights from cognitive psychology, decision analysis, and the scientific method. It helps analysts protect themselves from avoidable error, and improves their chances of making a correct judgment.
http://www2.parc.com/istl/projects/ach/ach.html

RapidMiner – About 6% of data miners use it – Can use R as an extension with a GUI
http://rapid-i.com/content/view/281/225/

R – 46% of data miners use this – in some ways better than commercial software – I’m not sure what the limit of this software is, incredibly powerful
http://www.r-project.org/

Network Mapping

Multiple tools – Finding sets of key players in a network – Cultural domain analysis – Network visualization – Software for analyzing ego-network data – Software package for visualizing social networks
http://www.analytictech.com/products.htm

NodeXL is a free, open-source template for Microsoft® Excel® 2007 and 2010 that makes it easy to explore network graphs. With NodeXL, you can enter a network edge list in a worksheet, click a button and see your graph, all in the familiar environment of the Excel window.
http://nodexl.codeplex.com/

Stanford Network Analysis Platform (SNAP) is a general purpose, high performance system for analysis and manipulation of large networks. Graphs consists of nodes and directed/undirected/multiple edges between the graph nodes. Networks are graphs with data on nodes and/or edges of the network.
http://snap.stanford.edu/snap/index.html

*ORA is a dynamic meta-network assessment and analysis tool developed by CASOS at Carnegie Mellon. It contains hundreds of social network, dynamic network metrics, trail metrics, procedures for grouping nodes, identifying local patterns, comparing and contrasting networks, groups, and individuals from a dynamic meta-network perspective. *ORA has been used to examine how networks change through space and time, contains procedures for moving back and forth between trail data (e.g. who was where when) and network data (who is connected to whom, who is connected to where …), and has a variety of geo-spatial network metrics, and change detection techniques. *ORA can handle multi-mode, multi-plex, multi-level networks. It can identify key players, groups and vulnerabilities, model network changes over time, and perform COA analysis. It has been tested with large networks (106 nodes per 5 entity classes).Distance based, algorithmic, and statistical procedures for comparing and contrasting networks are part of this toolkit.
http://www.casos.cs.cmu.edu/projects/ora/

NetworkX is a Python language software package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.
http://networkx.lanl.gov/

Social Networks Visualizer (SocNetV) is a flexible and user-friendly tool for the analysis and visualization of Social Networks. It lets you construct networks (mathematical graphs) with a few clicks on a virtual canvas or load networks of various formats (GraphViz, GraphML, Adjacency, Pajek, UCINET, etc) and modify them to suit your needs. SocNetV also offers a built-in web crawler, allowing you to automatically create networks from all links found in a given initial URL.
http://socnetv.sourceforge.net/

SUBDUE is a graph-based knowledge discovery system that finds structural, relational patterns in data representing entities and relationships. SUBDUE represents data using a labeled, directed graph in which entities are represented by labeled vertices or subgraphs, and relationships are represented by labeled edges between the entities. SUBDUE uses the minimum description length (MDL) principle to identify patterns that minimize the number of bits needed to describe the input graph after being compressed by the pattern. SUBDUE can perform several learning tasks, including unsupervised learning, supervised learning, clustering and graph grammar learning. SUBDUE has been successfully applied in a number of areas, including bioinformatics, web structure mining, counter-terrorism, social network analysis, aviation and geology.
http://ailab.wsu.edu/subdue/

A range of tools for social network analysis, including node and graph-level indices, structural distance and covariance methods, structural equivalence detection, p* modeling, random graph generation, and 2D/3D network visualization.(R based)
http://cran.us.r-project.org/web/packag … index.html

statnet is a suite of software packages for network analysis that implement recent advances in the statistical modeling of networks. The analytic framework is based on Exponential family Random Graph Models (ergm). statnet provides a comprehensive framework for ergm-based network modeling, including tools for model estimation, model evaluation, model-based network simulation, and network visualization. This broad functionality is powered by a central Markov chain Monte Carlo (MCMC) algorithm. (Requires R)
http://statnetproject.org/

Tulip is an information visualization framework dedicated to the analysis and visualization of relational data. Tulip aims to provide the developer with a complete library, supporting the design of interactive information visualization applications for relational data that can be tailored to the problems he or she is addressing.
http://tulip.labri.fr/TulipDrupal/

GraphChi is a spin-off of the GraphLab ( http://www.graphlab.org ) -project from the Carnegie Mellon University. It is based on research by Aapo Kyrola (http://www.cs.cmu.edu/~akyrola/) and his advisors.

GraphChi can run very large graph computations on just a single machine, by using a novel algorithm for processing the graph from disk (SSD or hard drive). Programs for GraphChi are written in the vertex-centric model, proposed by GraphLab and Google’s Pregel. GraphChi runs vertex-centric programs asynchronously (i.e changes written to edges are immediately visible to subsequent computation), and in parallel. GraphChi also supports streaming graph updates and removal of edges from the graph. Section ‘Performance’ contains some examples of applications implemented for GraphChi and their running times on GraphChi.

The promise of GraphChi is to bring web-scale graph computation, such as analysis of social networks, available to anyone with a modern laptop. It saves you from the hassle and costs of working with a distributed cluster or cloud services. We find it much easier to debug applications on a single computer than trying to understand how a distributed algorithm is executed.

In some cases GraphChi can solve bigger problems in reasonable time than many other available distributed frameworks. GraphChi also runs efficiently on servers with plenty of memory, and can use multiple disks in parallel by striping the data.
https://code.google.com/p/graphchi/

Web Based Stuff:

Play amateur Gestapo from the comfort of your living room:
http://littlesis.org/
http://theyrule.net/

Search Professionals by Name, Company or Title, painfully verbose compared to the above 2 tools
http://www.marketvisual.com/

Broad list of search engines

http://en.wikipedia.org/wiki/List_of_search_engines

&

http://www.wired.com/business/2009/06/coolsearchengines/

A tool that uses Palantir Government:
https://analyzethe.us

connected with the following datasets:
http://www.usaspending.gov
http://www.data.gov/
http://www.opensecrets.org/
https://www.epls.gov/
and some misc. others

Database Listings

http://www.forecastingprinciples.com/index.php?option=com_content&view=article&id=8&Itemid=18

http://www.datawrangling.com/some-datasets-available-on-the-web

http://datamarket.com/

Analytic Methods:

THIS BLOG IS PART OF CLASS PROJECT TO EXPLORE VARIOUS ANALYTIC TECHNIQUES USED BY MODERN INTELLIGENCE ANALYSTS (DELICIOUS ALL CAPS)
http://advat.blogspot.co.uk/

Morphological Analysis – A general method for non-quantified modeling
http://www.swemorph.com/pdf/gma.pdf

Modeling Complex Socio-Technical Systems using Morphological Analysis
http://www.swemorph.com/pdf/it-webart.pdf

CIA Tradecraft Manual

https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/Tradecraft%20Primer-apr09.pdf

Top 5 Intelligence Analysis Methods: Analysis Of Competing Hypotheses
http://sourcesandmethods.blogspot.com/2008/12/top-5-intelligence-analysis-methods_19.html
(the author scores a 4.4 of 5 on http://www.ratemyprofessors.com/ShowRatings.jsp?tid=545372 , 2.4 on the easiness scale)

http://en.wikipedia.org/wiki/Intelligence_analysis#Analytic_tradecraft
Many new analysts find that getting started is the hardest part of their job. Stating the objective, from the consumer’s standpoint, is an excellent starting point. If the analyst cannot define the consumer and his needs, how is it possible to provide analysis that complements what the consumer already knows.

“Ambassador Robert D. Blackwill … seized the attention of the class of some 30 [intelligence community managers] by asserting that as a policy official he never read … analytic papers. Why? “Because they were nonadhesive.” As Blackwill explained, they were written by people who did not know what he was trying to do and, so, could not help him get it done:
“When I was working at State on European affairs, for example, on certain issues I was the Secretary of State. DI analysts did not know that–that I was one of a handful of key decision makers on some very important matters….”

More charitably, he now characterizes his early periods of service at the NSC Staff and in State Department bureaus as ones of “mutual ignorance”

“DI analysts did not have the foggiest notion of what I did; and I did not have a clue as to what they could or should do.”[6]
Blackwill explained how he used his time efficiently, which rarely involved reading general CIA reports. “I read a lot. Much of it was press. You have to know how issues are coming across politically to get your job done. Also, cables from overseas for preparing agendas for meetings and sending and receiving messages from my counterparts in foreign governments. Countless versions of policy drafts from those competing for the President’s blessing. And dozens of phone calls. Many are a waste of time but have to be answered, again, for policy and political reasons.

“One more minute, please, on what I did not find useful. This is important. My job description called for me to help prepare the President for making policy decisions, including at meetings with foreign counterparts and other officials…. Do you think that after I have spent long weeks shaping the agenda, I have to be told a day or two before the German foreign minister visits Washington why he is coming?”

Replying To John Robb on Drones, Self-Driving Cars

John’s post:

I spent some time on the phone with a reporter from Inc. Magazine last week.  We were discussing the future of entepreneurship and where new opportunity could be found.

He asked me about drones and if there were opportunities for entrepreneurs there.

I told him that there were only two places where drones were going to gain traction:

  • Security.   From military operations to intelligence gathering to police surveillance.
  • DiY.   People building their own drones and finding interesting ways to use them.

That’s it.

Why?

All of the other uses of drones are closed off due to legal restrictions:

  • Drones for passenger transport.  It’s pretty clear that drones could be used to transport passengers safely and at much less cost than a manned aircraft.  It won’t happen.  Too many legal implications and push back from unions.
  • Drones for private info gathering.  Currently prevented.  There’s going to be legal wrangling over this for decades, which will prevent an industry from forming (other than “security” related).
  • Drones for short haul delivery/transport.  Too difficult to overcome the legal ramifications of operating drones on a mass scale near to homes/buildings.  It will definitely be used in the military.

Much of same logic is going to be applied to other forms of autonomous robotics.  For example: robots can drive a car better than a human being.  Google proved that already with their mapping car.  Will it be common to see “automated” cars in the next decade.  Probably not.  The first person killed by one will kill the industry through lawfare. Link

My response:

I doubt it John. I think you have over-weighted the power of lawsuits versus innovation.

Usually when there are legal blockages to technology, it holds the technology back for a short time until it finds a way around it. See the transition from embryonic stem cells to skin stem cells.

For cars in particular, people want safety in their cars much more than energy efficiency. That’s part of the reason SUV’s have outsold the economy car designs, most of the models are much more secure, barring the extremely large ones that are vulnerable to tipping over.

Recently smaller SUVs that combine the two attributes have become the most popular design. The demand for safety in automobiles has always been extreme, and in the case of the Google car Sebastian Thrun has been extremely careful in making sure the cars don’t have any accidents even in testing.

People put more trust into tech companies like Google than they do the legal system or congress, by a wide margin:
http://www.edelman.com/news/trust-in-government-suffers-a-severe-breakdown-across-the-globe/

Nevada has already legalized self driving cars, California is following.

So you assume that:
1. People’s desire for safety in consumer choices will be outweighed by their desire for control
2. There is no judo move to counter regulations, as there has been for many past technologies
3. That current friendly regulations are a smoke-screen for a coming crackdown
4. That people won’t trust the Google brand in particular in the case of the car
5. That a small number of lawsuits can destabilize a potentially multi-billion dollar industry, in spite of there being a perfect safety record thus far

Adjustable Taser/Pepper Spray Riot Shield By Bernardo Bajana

This shield takes several different concepts and chunks them into one. It’s close to a poorman’s exoskeleton. Most officer’s have a significant strength advantage over the people that do the rioting but an additional exoskeleton could be attached to provide short bursts of anaerobic power in repelling crowds. The filament that is used on this shield can be used to turn nearly anything into a touch taser.

Link

Mono Lake: Another ‘Triumph’ For Feminism In STEM?

“Late in 2010, scientists participating in a NASA news conference dropped a bombshell: they had found evidence that bacteria in California’s Mono Lake were metabolizing arsenic and using it in their life processes.
This was huge news, since arsenic is toxic for carbon based life. If some forms of life evolved a way to process it, this would open up a whole new field of biochemistry!…

However, almost immediately, the work came under attack. Biochemists accused the original team of not performing the research carefully (to put it delicately).

The original team, lead by Felisa Wolfe-Simon, has responded, saying they need to see a fully peer-reviewed paper before making up their minds.
I’ll note that emotions have run fairly high throughout this saga. Dr. Wolfe-Simon got a lot of attention…”
LINK

A LOT of attention. Before being refuted she was ranked in the top 100 most important people of 2011 by Time magazine.

Felisa Wolfe-Simon

Hmm. Hyphenated name. To be “equal” with a hubby? According to wiki, alma mater is Oberlin, a super liberal college more known more for the arts than its science programs; the type of enivronment where feminism is likely to be strong.

Some of the credit for this article must be given to Eric.

Nanotech in Nature: Gecko Feet

India Scores In Space

Google’s Director Of New Projects – On Innovation

Dr. Astro Teller goes over his system for creating innovation, pointing out that:

  • Innovation is counter-intuitive, otherwise we would already be doing it. This means that experts in a field usually will not be able to predict (1) how useful an innovation will be because they are so deeply entrenched in the system. (2)
  • Ideally you should find 3-4% of experts in a field you a innovating in to agree with you, look for the “weird” ones. They should be willing to join you in your project. If everyone agrees with you, you are too late. If no one agrees with you, you are wrong about it.
  • You can’t judge your feelings on it, you either get a metric to judge progress of an innovation or follow the inventor as he makes progress on his work.
  • It’s important that you have a story about how the innovation would effect people, so that you can create something that actually changes people’s lives.
  • A story has to mention the problem, a product or service that solves the problem, you have to show that there is something hard, be it in creating it or in the legal barriers, and show why it wasn’t already fixed by the market a few years ago.
  • He recommends making people also give him 10 bad ideas along with the one good one, to show that they have been exploring lots of little bets that could pay off, instead of just hinging their success on one “big idea”.
  • It’s also super important to figure out what tools you used to implement that idea, and then spend time on improving those tools so you can innovate and implement the idea even better the next time around.
  • He even goes as far as throwing out the code his team writes for the first 6 months, to give them a lot of room to explore a huge goal and then start fresh with what they learned without having to hook it on to an old system.
  • Rebuilding from scratch allows you to iron out the errors and correct dysfunction. If you can’t rebuild it, you didn’t understand it deeply enough in the first place.
  • Separate the people who have an emotional or practical stake in preventing innovation. You don’t want to start a popularity contest among people who don’t want you to win.
  • You can’t break too many assumptions at the same time, otherwise it will be too far outside of the range of most people to market.
  • He gets his engineer’s to say no more than yes. Engineer’s are emotionally attached to their projects, you need to get them to separate themselves from the project. He even provides an incentive for them to kill their own projects, offering them a bonus if the project works, no bonus if it doesn’t or a bonus +10% or more if they are willing to kill their project and start on something new. The cost of running dumb projects is huge compared to bonuses.
  • You have to even kill the good ideas so that the great ones can prosper.
  • If you want to give people a lot of room to succeed, you have to give them a lot of room to fail and make a mess.
  • Innovation tends to happen in small groups with lots of structure.
  • It can be easier to create a new version with exponential gains, instead of relying on incremental gains. It forces you to stretch your brain to find new ways of solving the problem. You already know you can’t get huge gains using traditional methods. (3)
  • The easiest way to fix problems is to transplant ideas from one field to another.
  • “Me too” ideas can make money, like making new iterations of facebook for doctors, dentists, ect… but they are not innovations. Innovations are a new class. (4)
  • The best fields for innovation are where people are torturing themselves. What is the most painful and awkward thing people are doing in spite of the challenges, how can we make that easier and then sell it?
  • Engineers are taught to solve problems, entrepreneurs and designers are taught to change problems. An engineer will create a great vase, a designer will create a “mechanism to display flowers”, to loosen up the constraints and find the best solution possible.(5)
  • Profit motives tend to drive success, even in social problems. Profit motives make the enterprise more sustainable. Profit motives and making the world a better place are not mutually exclusive.
  • Find areas where big problems exist and there has to be a better solution, even if you don’t know what it is.
  • Some things are on paths that are predictable, in other cases the future must be invented and cannot be predicted.

Most of the books on innovation were made by academics, and usually either written for housewifes or as a dry academic text. Teller does a better job in describing how to create a good incentive system for innovation:

1. Even when elites are aware of technologies they tend to downplay their importance, even the experts in the field didn’t estimate the changes wrought by gunpowder, automobiles or the pc. Tons of engineers had to change their skills almost overnight when the transistor overthrew the vacuum tube. The cybernetic steam engine governor was made by a kid who just wanted to play marbles. The elite’s curiosity wasn’t stimulated enough and they had too much emotional investment in the status quo.

2. Reid Hoffman: “A side note on invention and innovation: when you have an idea for a startup„ consult your network. Ask people what they think. Don’t look for flattery. If most people get it right away and call you a genius, you’re probably screwed; it likely means your idea is obvious and won’t work. What you’re looking for is a genuinely thoughtful response. Fully two thirds of people in my network thought LinkedIn was stupid idea. These are very smart people. They understood that there is zero value in a social network until you have a million users on it. But they didn’t know the secret plans that led us to believe we could pull it off. And getting to the first million users took us about 460 days. Now we grow at over 2 users per second.” Link

3. This is also called the Jack Welch Strech, See The Art Of Asking The Right Questions Link

4. It’s easier and more reliable to make money by copying other people. See First Mover Advantage v. Ecosystem/Fast Follower Advantage Link

5. Increasingly the field of marketing, engineering and design are merging into one as benefits are being custom tailored to products and distribution networks, see Airbnb Link

CIA Funded Method For Determining Political Instability

Taken from open source data.

The US Government-sponsored Political Instability Task Force presented many of its Phase V findings during a panel at the 2005 annual meeting of the American Political Science Association in Washington, DC, September 3, 2005. Copies of the three papers presented at the meeting are posted here in PDF format.

The PITF is funded by the Central Intelligence Agency. The PITF website is hosted by hosted by the Center for Global Policy at George Mason University and is provided as a public service. The views expressed herein are those of the Task Force and its individual members, and do not represent the views of the University or the US Government. Link

…First is trade openness (the total value of imports plus exports divided by GDP).
Countries with lower trade openness (at the 25th percentile in the global distribution) had roughly
two to three times higher odds of near-term instability than countries with higher openness to
trade (those at the 75th percentile). State-led discrimination reappears, but with a larger impact.
The odds ratio between states with and without major economic or political discrimination
ranges from three to forty across the three control sets. The large range suggests the presence of
outliers in control set B2, but the variable remains statistically significant across all three control
sets.

Colonial heritage makes a notable difference in stability, with countries that were not
formerly French colonies having odds of instability roughly four to thirteen times greater than former French possessions.

This most likely reflects the fact that France has been far more involved than other former colonial powers in
maintaining economic and political order in its prior domains, including supporting the West African Franc,
providing generous support to post-colonial rulers, and even intervening militarily to maintain unpopular rulers and head off rebellions.

We tested this argument with a categorical
version of a variable that counts a chief executive’s cumulative years in office and found that
new leaders (less than five years in office) and “entrenched” leaders (those more than fourteen
years in office) indeed faced higher odds of instability than their peers who had been in office from 5-14 years. The odds of near-term instability for short-term leaders were two to fifteen times higher, and those for entrenched leaders were six to twelve times higher.

Finally, we did find one effect of group composition on instability. Countries that had a
dominant religious majority (over two-thirds of the population identified with the main religious
group) were more likely to experience instability than countries in which the population was
more evenly divided among different religious groups. Countries with a dominant religious
majority faced relative odds of instability five to twelve times greater than those that were more
evenly divided

All of that said, regime type once again showed the strongest effects. With fewer cases
and thus smaller samples, we did not find significant differences among all five regime types;
instead, it was simply the case that full autocracies were most stable, partial democracies with
factionalism were the most unstable, and all other regimes fell in the same middling range of
instability. In particular, these other regimes had odds of instability that were six to nine times
higher than those of full autocracies. Here, however, the impact of partial democracies with
factionalism shoots right off the scale because in our data every African country that mixed
partial democracy with factionalism suffered instability”

Predictive Political Simulation Model – Senturion – Using Algorithms and Equations On Iraq, Palestine

Used to predict compromises and coalitions in political situations by modeling stakeholders. The report applies the model to Operation Iraqi Freedom, the Iraqi Elections in Jan 2005, and the Palestinian Leadership after Yassir Arafat’s death. All data fed into the simulation was open source.

http://www.ndu.edu/CTNSP/docUploaded/DTP%2032%20Senturion.pdf

h/t Justin Boland

Replying To James G On The Intelligence Gap

First the original post from Roissy/Heartiste:

1. How is the present automation and productivity conundrum qualitatively different than ones from the past (for example, the classic case of the auto replacing the horse and carriage)? If you do not believe it is qualitatively different, explain how we escape the “zero marginal productivity” worker trap, especially in an era when human capital is shrinking due to a combination of dysgenic birth rate differentials and mass migration of unskilled poor? Note: “Humans are fungible” is not an acceptable cop-out.

2. If, say, most of the profits go to the top 10% in society, while the bottom 90% are unemployed or marginally employed, how is it exactly that those top 10% will be able to extract profits from a customer base that doesn’t have the income stream to afford more than the basic necessities? Link

That’s a hell of a statement for anyone to make. Particularly if intelligence and training and the ability to create value that comes from it is being made easier and cheaper to obtain. If Einstein was born into the middle-ages and he spent his life as a banker, would his intelligence of been wasted? In which slice of history was our intelligence and ability to create value fully employed? With all of the resources at our fingertips, do you expect this 90% to just sit on their asses and not try to create something valuable? Or perhaps you expect a dramatic French Revolution ending to play out, with escape goats hanging from lamp posts? Do you really think that turning the US counter-culture into Hezbollah is anymore than dramatic masturbation that ignores the bloody reality of actual combat?

It was classic Tom Clancy stuff, all based on the idea you make war with stuff, not people. These guys just won’t face the fact that for the guerrilla, the key weapon, the only weapon that matters, is people—and starting a guerrilla war means sentencing most of the people in your address book to a very nasty death. Link

James G replies with Henry Hazlitt on how machines change value, you can read the entire piece here.

He then says:

Intelligence is efficient cross-domain optimisation; winning at chess requires efficient optimisation, but only within an extremely narrow domain. Chess is a restricted-domain problem, and a lot of man-hours have gone into developing chess AI, yet unremarkable humans managing computers can still lick an unaided machine.

Now consider a job that even the dullest human can do: supermarket cleaning. This is very cross-domain: it requires the ability to perform such diverse motions as walking, wiping, mopping, throwing, scrubbing and lifting. It requires the ability to communicate efficiently with humans. It requires the ability to change tasks on the fly. For sure, NASA could probably design a fantastically expensive self-cleaning supermarket; but what do you do if a kid throws up on the forecourt, or the roof starts leaking, or the stock layout needs to change? There’ll always be a need for human cleaners, at least until the advent of mobile human-level AI – which probably isn’t separable from the singularity, and a general end to this epoch of human civilisation.

I generally agree with this and think that a better, more correct definition of intelligence will include an individual’s resourcefulness. They will have to not only be able to find and integrate knowledge, but have good heuristics for validating it’s authenticity. “Who benefits?” must be asked constantly.

The new economy appears to be attention, and everyone is creating new and innovative ways of stealing that attention with emotional ploys for a variety of purposes, be it to secure a political campaign or to sell chicken sandwiches.

“this world is like three-dimensional chess, made more complex by the certain knowledge that there can be two things which are both true and yet which are mutually exclusive and contradict each other.”

But the interesting thing is that it’s getting easier and cheaper to close the intelligence gap. The bottom 90% may not have economic integration yet, but many of the important things that lead to value creation are being secured. The end game may involve the creation of a currency outside of the hands of a Nation-State, which isn’t that much of a stretch given how apathetic people are about their governments. Trust and legitimacy are two of the most important things in building financial relationships. At this point if there was an actual revolution in the US, people would be more likely to use GoogleBucks versus whatever the newly established state said was the currency. Any country that tries too hard to restrict the flow of vital information is guaranteeing it’s irrelevance in the modern economy, that’s the burden of an intelligence arms race.

Right now we’re doing very simple stuff with intelligence boosting, magnetic and electrical stimulation to promote cross-hemisphere communication, as well as searching for new nootropic drugs to boost performance. We’re also mapping out what parts of the brain are responsible for thinking,

The research team, led by lead author Shinji Nishimoto and professor Jack Gallant, used functional magnetic resonance imaging (fMRI) and computational modeling to produce videos of what participants were seeing — using only information about their brain signals. This may very well be the closest humanity has ever gotten to mind-reading.

Using functional magnetic resonance imaging (fMRI), the scientists gathered data on subjects’ brain activity as they were viewing scenes. In this case, subjects reclined in the brain scanner and watched 60 second clips of Hollywood movie trailers for several hours.

As the subjects watched, the fMRI machine analyzed their brain activity and produced something called the Blood Oxygenated Level Dependent signal, or BOLD signal for short. The BOLD signal is simply a measure of blood-flow to an area of the brain. This study focused on the brain area called primary visual cortex, which is responsible for early visual processing prior to consciousness. Link

There are also computer programs that have a limited effect like n-back (brain workshop can use the original Jaeggi parameters and is free) or speed reading training software. A lot of the computer programs are either free or low cost (or can be pirated).

This is from the unofficial n-back FAQ written by Gwern:

One of the nice things about N-back is that while it may or may not improve your IQ, it may help you in other ways. WM training helps alcoholics reduce their consumption28 and increases patience in recovering stimulant addicts (cocaine & methamphetamine)29. The self-discipline or willpower of students correlates better with grades than even IQ30, WM correlates with grades and lower behavioral problems31 & WM out-predicts grades 6 years later in 5-year olds & 2 years later in older children32. WM training has been shown to help children with ADHD33 and also preschoolers without ADHD34Lucas 2008 found behavior improvements at a summer camp. Another intervention using a miscellany of ‘reasoning’ games with young (7–9 years old) poor children found a Forwards Digit Span (but not Backwards) and IQ gains, with no gain to the subjects playing games requiring “rapid visual detection and rapid motor responses”35, but it’s worth remembering that IQ scores are unreliable in childhood36 or perhaps, as an adolescent brain imaging study indicates37, they simply are much more malleable at that point. (WM training in teenagers doesn’t seem much studied but given their issues, may help; see “Beautiful Brains” or “The Trouble With Teens”.) Link

The thing about nootropics is that they are very cheap, each generally costing less than $100 for 3 months of doses. You can build your own brain stimulator for around $30 or less. Modafinil will run about $80-250 for a month’s supply, though it could be substituted for cheaper nootropics from the -racetam family of drugs. They aren’t a miracle drugs, but they are better than nothing.

Genotyping is $299 right now, which can provide you with your genetic profile and a list of diseases you are susceptible too, you have to be pretty jaded to think that isn’t a major advancement (though the treatment costs obviously vary by country, see medical tourism).

Most of these nootropics are useful for at least slowing the effects of Alzheimer’s (Piracetam is the most well documented), and an ounce of prevention is worth a pound of cure here. We haven’t seen the creation of super-expensive brain enhancements that would divide the rich and the poor, not if the poor here in America can afford iPhones and Air Jordan’s. The major exception for America is healthcare, which is a highly debated topic being milked for all of it’s political capital.

You should be able to up your raw IQ by at least one deviation with a combo of the above stuff, the guys at bulletproof exec used n-back and got 18 and 12 points boost respectively. You can’t buy any of this with foodstamps but the barriers to increasing intelligence is going way down. Access to knowledge has never been easier as well: edx, Khan academy, JOVE, Wikipedia and thepiratebay have made sure of that.

One item of interest: imagine what happens when you apply the chess tactical aid to science, I.E. having an AI design and perform experiments aided by a team of human scientists, keeping in mind that we’re getting closer to a real brain-machine interface. We can’t be sure of what the brain-machine interface or anti-aging treatments would cost, but doesn’t mean it would widen the actual gap between the rich and poor.

Nooks/Kindles Reveal What People Read, How Quickly & How Many Pages

Governments are too inefficient and behind the curve to execute a real 1984 type society. Corporations, particularly technology/internet ones, are the kind that have the engineering know-how to create the real big brother. That’s why the government bought Palantir’s products, the best engineers don’t work in government anymore. In a real martial law situation I would expect massive privatization to cover the gaps. Even Wal-Mart is using former intelligence officers now. This of course doesn’t cover the “black projects”, but no one can exactly put their finger on where that money goes and what comes of it.

http://consumerist.com/2012/06/barnes-noble-amazon-know-which-sections-of-fifty-shades-of-grey-youre-reading-over-and-over.html

Data collected from Nooks reveals, for example, how far readers get in particular books, how quickly they read and how readers of particular genres engage with books. Jim Hilt, the company’s vice president of e-books, says the company is starting to share their insights with publishers to help them create books that better hold people’s attention.

Peter Thiel and George Gilder debate on “The Prospects for Technology and Economic Growth”

Fingerprint Reader Captures Prints From 6 Meters Away

HUNTSVILLE, Alabama — Forget the key card to your office building? Just wave your hand at the door, and you’re in. “You don’t have to stop at a station. Nobody checks your ID. You just walk through,” explains Clemson-educated physicist Joel Burcham of his new Huntsville company called IDair.

IDair makes a machine that Burcham says can photographically capture a fingerprint from as far away as six meters in enough detail to match against a database. Add facial and iris-recognition technology, Burcham said, and you have the basis for a good biometrics system that can control access to any building or room within a building. Link

How Planned Obsolesce Works – Faulty Capacitors

“These known-defective components are found in virtually everything electronic-related. Nearly all electronic devices have electrolytic capacitors in them – the purpose of these devices is to smooth electrical currents, much like a shock absorber does in a car. Without them working properly circuits either don’t work at all or become unstable.

If you’ve ever heard a “snapping” noise from some piece of electronic equipment, shortly before or concurrently with it ceasing to work, you almost certainly have run into one of these known-defective components.

Yet despite knowledge of the fact that these things are in literally everything – phones, computer board, power supplies, TVs, monitors and other products – there has been no recall that I know of issued. Link

“These are not accidents or “ordinary wear and tear” failures.

You are being screwed, America, by manufacturers who produced products with defective components that are necessary for both the function and safety of the devices you’re using.

These components will fail, your consumer electronics will stop working long before they should.

This is not about warranties. These capacitors are not merchantable. They are unfit for any purpose, as they are known to be defective and as such it is known that they will fail, destroying your device.

It is as if these products have a built-in time bomb ala “Mission Impossible”, that just happens to go off shortly after your warranty expires.” Link

Two Contrasting Views On Future Warfare

The State will have become technocratic – fascism by remote control – the dream of control, coveted by evil men for generations, will have come to fruition. We have a scant few years to arrest the development of these technologies or to rearchitect the social foundations of liberty to survive a situation where combat robots leave the population largely powerless to resist tyranny, whether they have their rifles or not. To develop the technology to defeat the rifle utterly in the field is roughly equivalent to absolute, final, global disarmament of the population.

I am not suggesting that these combat robots will have human intelligence. I am not suggesting they will be effective policemen in the crime solving sense. They will start as remote controlled weapons platforms, then evolve common sense on navigation, then target selection, then tactics and strategy. What can be automated successfully will be automated, and the rest will be left to men in bunkers viewing screens where blood is rendered in black or blue, not red, and the faces of the fallen are fuzzed out as distractions from the real work of identifying and terminating enemies among the living.

….

So what do we do about this future of oppression at the hands of robots developed to defeat the improvised weapons of freedom fighters, revolutionaries and insurgents everywhere? What do we do about a future where little guy finally has no chance at all against the State, should the state turn against him and seek to drive him to the wall? What indeed can we do about that situation?

I want you to get serious about putting aside your political differences about the economy, and to get serious, left and right, about making sure that our children don’t grow up in a world where men they will never see or vote for control the box on the corner that tortures you with an invisible ray any time you get out of line.

I will note that the development of a remote mass torture device contravenes all human and natural law, and the insistence that it will “save lives” is based on a simple misunderstanding: if the people are taking to the streets and screaming for change, and you torture them where they stand to make them stop asking for change, eventually they will turn to real violence and kill the hand that tortures them to make them comply, if they can. And if they cannot kill that hand, what has been created is a hell: a torture state which one cannot overthrow or escape from. What hand will wield these torture machines in a few generations?

Lines are being crossed here. Technologies which stand every chance of enslaving us all are being developed to win the Iraq war, because the Iraq war is typically of armed resistance to government anywhere. It is disorganized, angry men with rifles and bombs dying for what they believe in, however misguided. If the ability to defeat such groups is developed and placed into the hands of the current incumbent governments and power groups, the same processes that gave rise to a free revolutionary America will no longer operate, and there will be no more stands to be made against the Empire. Those who stand will be ceaselessly and cheaply cut down by replaceable robot warriors manufactured far from the fray, operated from bunkers, and deployed far away from TV cameras. There will be little or no home front pressure to stop unjust and unnecessary wars because only the blood of the enemy will be shed in armed conflicts. The human cost of war will be borne entirely by the underdog, and therefore the underdogs will have lost their primary means of making the incumbent power groups change course. And, let me tell you, we are all potentially that underdog.

Link

But note that current drone crews do suffer from PTSD, they are far from being dehumanized. Drones may also change how the battlefield terrain is shaped through construction.

But you’re in luck, all you thin healthy smiley bastards: I don’t think it’ll happen like that. It’s going to be weirder, slower, and a lot less Star-Wars-ey than people think.

It’s easy to get all excited about blasters, space battles, lasers and all that Luke Skywalker stuff. But my job is to give you my best guess on what’s really gonna happen. And you know, I’m not even sure war will survive. War seems too good for people like you: you beach volleyball people. You’ve made getting healthy and thin a religion, so why would you want war? Well, one thing: it won’t be the cool sci-fi war you like to think about, you saving Carrie Fisher from Jabba with your Jedi mind crap….

Spacewar — Killer satellites, orbital lasers…won’t happen. Nothing but lame NASA fundraising ideas, cooked up by corrupt lobbyists and corporations that make a living off the federal budget. Never convinced anybody this side of Newsweek. 150 years from now there’ll be nobody on the moon, nobody on Mars — just some fragile, expensive tools floating up there, not worth blasting, far too expensive to risk.

By now we can keep a Predator RPV hovering week after week, waiting for a target. When they finally persuaded the USAF to give the Predator RPVs a chance in Afghanistan, they had to admit the damn things worked even better than their advocates were promising. They’re amazing: too small to spot, damned hard to shoot down, and cheap enough that we don’t lose much even if it does get hit. And you know the best thing about RPVs? They don’t react to torture. No pilot to go on Iraqi TV looking like Jake LaMotta after twelve rounds with Sugar Ray Robinson and start apologizing for disturbing Baghdaders’ beauty sleep.

The trouble with this nice clean automated-war scenario is that nobody wants to play with us. The US can play that game, but who else can? The Israelis? They’re the only real combat-tested RPV-using army. And if it came to a US vs. Israel war, let’s face it: the US Congress would back Israel all the way, and the US’d have to surrender before a shot was fired.

Try plugging the hi-tech, RPV-heavy war plan to a more even-sided war: say, an all-out struggle for world domination between the US and China ten years from now. The first thing you realize is that it’ll come down to production rates. You’re gonna lose a lot of hardware in a hurry. Like aircraft in the early days of WW I, RPVs will go from surveillance to attack, and that will lead to interceptor models designed to destroy enemy RPVs. There’ll be unmanned dogfights, and since these things are easy to make, the dogfights will be unbelievably massive, maybe hundreds of thousands of individual combats in the sky over the battlefield. It comes down to our factories vs. theirs. If you can replace it faster than they do, maybe you win. It’ll all be as harmless as a nerd picnic on the school field Saturday afternoon, with the Asian kids and the pasty white kids each piloting their little remote-controlled MiG’s and F-16s and arguing about who killed who, then going off for pizza.

As the two-tier war system develops, the hi-tech nations won’t even associate war with death any more. War will be a demolition derby: our machines beat your machines. Nobody has to die. When the dogfight between a Chinese and an American RPV finishes, nobody will die; the US controller will disconnect from his monitor and have a beer, and so will the Chinese.

Production dominance will tilt one way or the other, at last: you own the skies. They can’t send up any more RPVs, and you can. OK; you’ve won. Now what? Do you start carpet bombing their cities? What the Hell for? The civilian population won’t even matter any more. Kill a hundred million Chinese — so what?

because an ICBM isn’t the only way to deliver a nuke.

I mean, just think for a minute. You’re Mao. You hate the US, you have a few big ripe homemade nukes, and you want to be sure the Americans know they can’t push you too far. Do you build ICBMs? Sure, a few — enough to keep the Japanese and the Russians awake. But you don’t really trust those homemade missiles. And — this is kinda the key point, so lissen up here — you don’t need to. Because a regime like Mao’s (or Stalin’s or Kim Il Sung’s) does one thing really, really well: spy stuff.

And if you’ve got a good spy service, delivering nukes is a cinch. A pickup truck is a perfectly effective way to deliver a nuke. How many pickup trucks cross from Canada or Mexico every year? Every day? You think every one of those gets searched? How many ships call at US ports every year? How many get really carefully searched? How hard is it to carry a nuke to an American harbor in a harmless-looking Liberian-registered cargo ship, then dump it over the side somewhere near the East River, or the Bay Bridge?

Why bother killing a few million civilians? Won’t settle anything. Just makes you look bad. In fact, nothing seems to be on the line anymore — not people’s lives, not even their jobs, no matter how much they fuck up.

The only enjoyable wars will be the mismatches, when the machine armies are unleashed on the savages. We’ve seen some of them lately: the NATO air forces working out on Serbia, the US and British planes playing with the Iraqis like a couple of kittens with a half-dead mouse. They’re the wars people will enjoy, because the targets are so easy, so undefended, that there are lots of good gun-camera shots.

But these wars have a little weakness: they never solve the problem. NATO killed a few thousand Serbs too stupid to realize their fellow Christians didn’t give a fuck about them. And the Serbs pulled back. But the Albanians moved in. You go into a slum like the Balkans, try to fix things up by slapping around one gang — and the gang next door comes in, kills their families and takes their houses. It’s embarrassing. From what I hear, a lot of NATO soldiers dream non-stop of the day they’ll be allowed to fire on the Albanian thugs they’re supposed to be protecting.

The answer is obvious: annihilation. The two-tier wars will get really annoying. How many times do you go in (and “you” could be the Chinese, the Indians, or whoever’s running the show 100 years from now) and separate these drunken smalltime thugs? Sooner or later somebody will suggest the neutron-bomb option. Nothing dramatic, just a Raid commercial on a larger scale.

They’ll be provoked. That’s a sure thing — before the ruling countries take the annihilation option, they will be HELL OF provoked. The lower tier will have one weapon: the willingness to die and to kill. You don’t need hi-tech to kill a lot of people. You think Mohammed Atta could pass a course on jet engineering? Physics? He couldn’t’ve got into Solano Community College, and all you need to pass there is two-thirds attendance. The loser countries, the ones who can’t do math, are gonna skip shop class, skip the machine crap, and go back to basics: kill a lot of people. They’ll do Columbine on a worldwide scale. All the losers will come to the lobby with guns. Serbs, North Koreans, Tamil Sri Lankans, will walk into the lobbies of the machine peoples’ towers like Keanu in The Matrix. They will splatter those security guards, they will smash up the decorative marble, they will disrupt office routine with drums of gasoline and vials of pesticide and rerouted sewage floods; they will turn the cities against their citizens and kill, kill, kill.

And the upper tier will respond. They’ll be patient. They’ll endure the first twenty or so urban massacres in a civilized way. Then they’ll think of the obvious: the Raid solution. Every pesticide commercial they ever saw will occur to them as they decide what to do with the Haitians, the Tamils…and finally somoeone in a government office in Beijing or Washington or Delhi will decide to do something permanent about the vermin. Ah yes, the Balkans: nice country. Too bad it’s infested with two-legged varmints. Why not clean ‘em out? It’ll strike somebody as a good idea, sooner or later. And that’s how we’ll have our first nuclear war: not the old Cold-War scenario where two nuclear-armed nations wipe each other out, but a perfectly logical one-sided version: China, or India, or us, or whoever, will simply sterilize the Balkans. (Or Java, or the South Bronx).

Link

So key questions right now are:

  • What countermeasures can be deployed versus drones? (Electronic warfare, jamming, masers, drone leader targeting ect…)
  • If the focus is exclusively on electronic elements, what about other spheres that are under-utilized?
  • Will the Nation-State exist in it’s current form, despite the crisis of legitimacy and economic disincentives?
  • Will the economy of scale favor large or small operational groups?
  • Will the production of drones favor mass drone warfare, or a mix of quality and quantity?
  • How will production facilities balance quality and quantity?
  • What is the size of the organization best suited for mass production, refinement and deployment of drones when factoring in automation?
  • What vulnerabilities exist in the all important supply chain?
%d bloggers like this: