[Synthesis] Advice To The True Seeker

“No theory, no ready-made system, no book that has ever been written will save the world. I cleave to no system. I am a true seeker.” - Mikhail Bakunin

While I didn’t know it when it was happening, for most of my childhood my father was teaching me the art of seeking secrets. He was a thrift store hound always on the hunt for a better deal, and made a significant portion of his living reselling pawn shop stock on Ebay. He’d sit me down at the computer with him where he’d scope out a rare guitar on some stores website for much less than it was worth. There was a life lesson for me there, he said that if he posted the existence of that guitar on craislist it’d be gone within the hour. But it was still there because nobody knew it was valuable. I tell you this instead of exploiting it myself because it’s no longer there to be exploited: Pawn shops wised up years ago and started googling the going price of their stock on Ebay so as not to lose out.

Still, my father was always looking to get rich quick. Always looking for the clever scheme that would take him to the top. While I never entirely bought into his ideology that secrets are abundant and the right one will make you rich, I did inherit his sense that hidden knowledge is possible to possess. It wasn’t really clear while I was growing up because I had no frame of reference, but my father is a profoundly strange man who has many traits of what we would today call autism. Stereotypically, autists have a special interest and my fathers was money. Every hobby he had, every conversation and every idea almost always came back around to money or music. He liked taking me to the park, but on a rotation for the parks which can be metal detected for coins. Metal detecting was irresistable for him, the idea that you can wave a magic device over the ground and find wealth that no one else would underneath. He’d regale me with tales of the hidden treasures that were rumored to be buried deep in local fields that someone else had put there for a myriad of reasons.

Ultimately, my father is a tragic figure. He was obsessed with Bill Gates, back then the wealthiest person in the world. He would point out that Bill Gates goes to the bathroom and makes more money than we probably do annually. The tragedy is that while he was given an extraordinary passion for the almighty dollar, he was not given the temperament or lust for the insight that might let him get it. My dad’s thirst for knowledge extended only to the bounds of what was easy, what might let him dream about one day being rich. While it might make an amusing hobby, it seems almost certain that my Dad was losing money on the opportunity cost of prospecting for coins at the local park. His desires outstripped his mindset.

Insight

One of the most interesting ideas in information theory is that prediction is a form of compression. If you have a magical box which can play you the last 90% of a song given the first 10%, then you only need to store the first 10% of a song to play it. The amount of space required has shrunk by a factor of ten. To put it another way, if you can predict an entire set of symbols from a fraction of them then you’ve compressed the symbol set. This is something like how knowledge works. The most powerful forms of knowledge, usually called wisdom or insight, allow you to predict a large number of lesser ideas which have the insight at their base. They’re almost literally higher up in the food chain than what lies below. Often though they require effort to truly understand, to get a feel for their scope and limits. Without such shortcuts the world we find ourselves in would be nearly impenetrable to human reason. A short list of such shortcuts is here, but there are many many more besides.

We can only speculate when it comes to the true effectiveness of insight, but I would be genuinely surprised if it didn’t perform better than garden variety knowledge. Humans spend so much time trying to distill ideas to their essence, in pithy quotes and witty jokes, public speaking engagements and brilliant explanations. Teaching as a field would be solved by now if there were no work left to be done in knowledge transmission. (Which as Shannon tells us is intimately tied up with the business of compression and prediction.) One of the ultimate goals of Computer Science is to teach this human ability to computers, to look at data and notice patterns, to predict the next item in the sequence. Machine Learning for example is all about getting a computer to predict new things based on what it’s already seen. The famous Raven’s Matrice IQ test literally asks respondents to prove their intelligence by predicting the next item in an observed pattern. If insight isn’t useful, at the very least humans seem believe it is.

In her essay “Trying to See Through: A Unified Theory of Nerddom”, Sister Y contrasts insight with humor. It might not seem obvious at first, but humor and insight are inverse disciplines. Insight is about pattern fitting, but humor is about anti-fitting, the subversion of expectations. For years now our notion of mass political understanding has been driven by humor. Family Guy, South Park, Saturday Night Live, comedians such as George Carlin, political cartoons and other political humor have been the main vehicle for the average person to understand the political world around them. Our recent shift towards conspiracy is concerning on multiple levels, but one hopeful interpretation is that it represents a shift towards insight. If SNL is political humor gone mad, then conspiracy is insight pattern matching gone wild. My own political taste started with conspiracy but is now much more discerning.

Secrets

Hidden knowledge is one of the great fascinations for humanity. From the stage magician enviously asked how he performs his miraculous tricks to the petty gossip exchanged between schoolyard children and nosy neighbors, from the philosopher pondering the meaning of the divine to the scientist dutifully observing our natural world for clues to its structure there are few men and women on this earth with no interest in secrets. While we all share a common interest in hidden knowledge, we’re vastly differentiated by taste. Perhaps the lowest level of hidden knowledge is gossip, certainly people consider it the province of the contemptible. Tabloid magazines which focus on the distant lives of rich and famous people peddle hidden knowledge of no real importance for the world or the voyeurs who consume it, and probably come by their contempt rightly.

In his essay distilling Peter Thiel’s startup class lesson on secrets, Blake Masters distinguishes between social secrets about people and natural secrets about the world around us. Social secrets have a poor reputation among some folk as the vice of petty and uninspired people, and natural secrets have an equally ill reputation among those possessed of the notion that they are the domain of those cold and lifeless. Neither is true, and the Renaissance figures that strived to combine naturalist and artist into a composite lifeform should be inspiration to us all in our endeavors. Leonardo Da Vinci would be ashamed of both parties.

The secrets about people most attractive to the intelligent are the insightful ones. Not the individual concerns of individual people, but ideas that allow patterns to be seen, trends to be identified in society. Personality tests, Psychotherapy, Psychology and mental pathology, Anthropology, all come from a place of seeking higher knowledge about humanity itself. “What does it mean to be human?” is a question haunted by cliche, but as we gain more and more control over our physical and mental form it only grows in importance. The individual concerns of individual people are important too, but insight lets us handle them more effectively and more efficiently.

A great deal of my affection for hackers comes from the unique way they bridge the world of seeking secrets about people and secrets about the natural world. This might seem strange, since the stereotype is that hackers are lonely people that are alienated from others, but this is only half truth. In both the open source MIT tradition and the computer intrusion phone phreaking tradition, the search for secrets and excellence are paramount but fellow travelers are absolutely welcome on the journey. Further, much of even the ‘benign’ hacking tradition relies on the manipulation of social reality, the invisible relationships between people and symbols and things that are obvious to us but might confuse a visitor from Mars. For example, this story from the Jargon File about sneaking a computer into a hospital exemplifies the nature of social reality well. In Sister Y’s essay she hypothesizes that nerds are people who have a natural ability to see the underlying mechanisms of social reality in a way that is invisible to most people. Mostly through their natural inability to understand it in one way or another. Things that normal people take for granted confuse nerds, which provides the impetus for making discoveries about social reality itself.

Caution

In his book The Hacker Crackdown Bruce Sterling writes:

The pure technical sweetness of the Bell System gave its operators, inventors and engineers a deeply satisfying sense of power and mastery. They had devoted their lives to improving this vast nation-spanning machine; over years, whole human lives, they had watched it improve and grow. It was like a great technological temple. They were an elite, and they knew it -- even if others did not; in fact, they felt even more powerful because others did not understand. The deep attraction of this sensation of elite technical power should never be underestimated. "Technical power" is not for everybody; for many people it simply has no charm at all. But for some people, it becomes the core of their lives. For a few, it is overwhelming, obsessive; it becomes something close to an addiction. People -- especially clever teenage boys whose lives are otherwise mostly powerless and put-upon - love this sensation of secret power, and are willing to do all sorts of amazing things to achieve it. The technical power of electronics has motivated many strange acts detailed in this book, which would otherwise be inexplicable.

And so it is with all insight. Some forms of secret and pattern matching are absolutely intoxicating to a select few and utterly barren to everyone else. Some find the entire business of insight to be headache inducing, or boring. Perhaps a bit of both. The true seeker then is likely to be lonely, seeking something which is esoteric by most peoples standards and constantly leaving in search of ever greener pastures. There are many traps lying in wait for a naive traveler seeking secrets and insight. I’ll outline them now so as to hopefully inoculate you at least partially against them.

Mysticism

In statistics there is a maxim which says in casual terms that the more variables you analyze at once the more likely you are to find a spurious relationship by coincidence. When you move enough variables into one analysis it just becomes junk. Certain branches of thought utilize this trick to come to whatever conclusion they please under the guise of hidden esoteric knowledge. The story Unsong by Scott Alexander illustrates this well. While I won’t necessarily warn you away from them on this basis, you should be aware that most of the things in the ‘esoteric secret’ bin are junk with no redeeming value.

Blake Masters immediately dismisses esoteric secrets in his Thiel startup class essay, which was probably premature. Many of the things we now take for granted as respectable fields of study started out as esoteric secrets. In the Soviet Union for example ‘Computer Science’ was long considered pseudoscience by the managerial class. You might think that is the Soviet Union and your society is surely better, but in reality your society probably just has a different set of biases that blind you to the ignored fields of inquiry sitting under your feet. Esoteric secrets are a high risk high reward endeavor, and dismissing them out of hand so quickly is a failing of Master’s essay.

Conspiracy Theory

I touched on this earlier when describing insight, but conspiracy theories are a common trap for the person that knows there are secrets to be sought, but isn’t quite sure what criteria to use for evaluating them. You’ve probably seen The Matrix and identified with the protagonist. What you have to understand is that The Matrix skips the most important part of things, figuring out what is and isn’t worth investigating. While it’s a great movie, and still my favorite to this day it’ll seriously mislead you in figuring out how to really start looking for “glitches in the matrix”.

Conspiracy theories often involve at least an element of mysticism, because they usually rely on putting many variables together into a narrative. One of the things which will help you out here is the notion of selection bias, which here essentially says that if you take all the variables in the world and run them together looking for interesting coincidence you’ll find it. Often such theories are built on a kernel of truth which is reinforced through various errors in data collection and reasoning. In her book “The Secret” Rhonda Byrne insists she knows the secret works because it’s in line with Quantum Mechanics, and she knows it’s in line with Quantum Mechanics because the secret allowed her to read books on such with no education and understand them. Of course, more likely than not Byrne just convinced herself she understood what she did not, which in complex technical subjects is really quite easy.

I fell into this trap when I was twelve, and a lot of my current thinking patterns are a result of the necessary mechanisms I had to come up with to get out of it. I now think of things in terms of two distinct axis, plausibility and probability. Plausibility is about the structural integrity of an idea, whether it’s self consistent, uses sensible reasoning, violates basic postulates about my world like “the sky is blue”, etc. Probability is about whether we actually live in the world where the thing is true. For example, there’s nothing intuitively wrong with the idea that the world is flat, it’s through careful observation of our real world that we realize it’s very unlikely to be the case. We could live in the world that is flat, but we know we do not because the things we observe would not be the way they are if that were the case. The reason why it’s probability specifically is that you don’t really know anything for sure, in the end you could always be inside a simulated reality, or insane, or some other very remote but still entirely possible way you could be wrong. Things only become so likely as you gain evidence for them, but no more likely than that. Consider for example that you’ve probably never really done any independent verification that the earth is round, you probably just took it on faith when others told you. If you react to this knowledge by deciding it’s plausible the earth isn’t round, you’re probably making a mistake because a really significant fraction of your knowledge about the world is based on things you’ve never empirically verified.

Guru Writing

There’s a certain kind of essay style which is fairly popular but as far as I know doesn’t have a name identifying it. I prefer to call it guru writing, because it takes as an assumption that the person writing has worthy advice and should be emulated in their conduct. Authors who consistently produce essays in this fashion I like to refer to as guru writers or guru authors. Some examples include Ramit Sethi, Steve Palvina, Eliezer Yudkowsky, and Paul Graham. The primary thing that distinguishes a guru author from other forms of advice column and the like is that a guru author essentially writes their worldview explicitly. Reading their writing is a way of accessing their worldview and helping you think like them. Some people have powerful worldviews and their guru writing is helpful, others are basically scammers looking to make a quick buck. When you’re just starting out, good guru writers can seriously boost your thinking and show you territory you probably wouldn’t explore on your own. Reading them is a good way to get a lot of insight quickly, but it also comes with the downside of potentially opening yourself up to scammers and bad advice. The solution to that of course is to maintain a good separation between yourself and the author so you can evaluate ideas critically.

As you get more experience you’ll probably become less tolerant of the excesses guru authors make, and you’ll probably want more standalone writing with a humbler tone. All of this is available, and you’ll get it from academics and field specialists and people who are too skeptical of their own worldview to sell you on it entirely. But before that happens you’ll probably enjoy the writing of people who share the most potent secrets and insights making up their worldview at a mile a minute.

Links

This piece is a synthesis of multiple authors ideas with my own. I encourage you to read the original essays which inspired this one.

Masters, Blake. (2012, May 11). Peter Thiel’s CS183: Startup - Class 11 notes essay. Retrieved from Link

Sister Y. (2012, Sep 18). Trying to see through: A unified theory of nerddom. Retrieved from Link

Lucky, Robert. (1991). Silicon dreams: Man, information, and machine. St Martins Pr, NYC.