tag:nikolayid.posthaven.com,2013:/posts nikolayid 2020-06-11T15:36:57Z Nikolay Dubina tag:nikolayid.posthaven.com,2013:Post/1557897 2020-06-11T15:35:57Z 2020-06-11T15:36:57Z new-blog

Hi, if you are reading this, you should probably go here https://nikolay.dubina.dev

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1555682 2020-06-07T12:53:05Z 2020-06-07T15:00:43Z the-go-bits
I recently finished "The Go Programming Language". It is very well written, in-depth book. The learning curve is very smooth, I think even relatively new programmers may use this book as intro to their first programming languages. Otherwise, check out O'Reilly "Introducing Go: Build Reliable, Scalable Programs", which I think is much friendlier but covers less.

Here is a raw list of some interesting bits that I learned primarily from "The Go Programming Language" I collected for my future reference.

* there are constants, variables (addressable values) and non-addressable values (e.g. anonymous structs, pointer to return of a function). Variable is something that stores value and has a name (e.g. x) or can deduce name from the left hand side (x.foo or x[i])
* lexical scope — piece of code where symbols can collide and resolved (function body, if, for, switch statements, package)
* can override variable names in nested lexical scopes
* package is the most outer lexical scope
* lifetime of a variable != lexical scope

* there is no distinction between heap and stack
* operator address-of "&" can be applied only to variables
* pointer is an address of a variable. All variables have addresses and can have pointer, but not all values have pointers
* pointer is not a numerical address of OS level memory
* pointer can be used to compare to null, compare to other pointer, dereference, or get address of variable. There is no pointer arithmetic
* it is perfectly fine to return address of a local variable in function
* function "new" creates new unnamed variable and returns its pointer (it is same as returning reference to local variable in function)

* swapping variables: "i, j = j, i" also can do "i, j, k = j, k, i" etc.
* flag package has nice API for arguments used to run program

* if package has "init" function it will be run once for go program when it is first time imported
* numeric types compare only within type (1.(int64) != 1.(int32))

* all Go source code is in UTF-8
* strings inside Go code are string-literals, and they are in UTF-8 too
* type string = byte[]
* UTF-8 is multi-byte encoding that is fully compatible with ASCII. It can represent all languages and more. It is efficient and can represent a lot of numbers. Sorting UTF-8 by bytes is lexicographical sorting. It has a number other benefits too. Few of co-authors of Go are co-authors of UTF-8. There is also UTF-16 and UTF-32, they are simpler but less memory efficient.
* use runes to work with UTF-8 strings
* use bytes[] for non UTF-8 strings, handle encoding yourself of by 3rd party module
* modules strings and unicode are dealing with UTF-8
* watch out for filepath, since filepath has to be in encoding of OS, which can vary. For example it can use Korean or Japanese non UTF-8 encoding

* constants are untyped, which allows to use "1" and determine at compile time if it is int64, uint64, float, etc. 
* constant expression iota allows to iterate in definition of consts. Useful for enums and bitsets
* numerical constats can be very large. It is safe to assume that it is 512 bits long. 
* constants do checks at compile time for division on zero etc. So that you always get a usual number in at compile time.

* [...]int64{1,2,3} notation is nice for arrays that you don't want to count how many elements it has
* slice points to element in array, to size of elements and capacity. Very safe and space efficient structure.
* to append to slice use "x = append(x, y...)" where x,y are slices. "y..." is expansion of slice into list of arguments.
* use "func x(args... int)" for variable number of arguments
* often useful not to "pop" from slice, but just to reduce slice, and consider values beyond slice as garbage. Very convenience since no need to explicitly free memory.
* maps are not guaranteed to have order, sometimes go runtime changes order from run-to-run of same code
* in maps key should be comparable with "==", which is a lot of types, even functions!

* functions are first class in go, meaning they can be passed as arguments or assigned to variables
* closures are possible. closure is a function with access to the scope they have been defined in
* function variables can be compared by their pointers

* can use shorthand to defined multiple fields with same type in structs "type A struct {a,b int}"
* structs literals can have named fields form like "x := Point{X: 1, Y: 2, name: "asdf"}" for publicly exposed fields
* Promotion mechanism. Stucts can be embedded with like "type Window{Box, size int}" where Box is a type. This will result in creating public field Box and syntactic sugar to refer to methods of Box field, so no need to chain methods of fields.

* type error = string
* error interface is single function "Error() string"
* if you have logic to check if error is specific type of error by using type assertion (like EOD error), you need to do that immediately up the call stack. This is because "fmt.Errorf" is used often and it takes error and makes a string from it, which discards previous concrete type and you would not be able to do type assertions anymore. TODO: what if you still want to? 

* methods have resolution mechanism. If receiver is pointer, it can be used with variable. If receiver is variable, it can be used with pointer. And of course it can be used with same receiver type it was defined with

* bitset implemented with numbers is nice, since you can have fast union and other bitwise logic.
* interface containing a nil pointer is not nil. So if you have pointer to interface, then it will never be nil. This is because interface internally contains interface value and interface type, and interface type is not nil always.

* there is a sort package that can be used to sort with arbitrary compare function. To use it need to define a function that returns new type that implements sortable interface (Len(),  Less(i,j), Swap(i,j)).
* there is type-switch notation "switch x.(type)" or "switch x := x.(type)" if you want to use x with that time in switch scope

* there is main goroutine that starts with main program
* goroutine has variable stack associated, OS threads have 4MB, goroutine can have as low as 4KB and as much as 1GB, which allows for very large number of goroutines and very deep recursive calls

* channels are useful when they are buffered and no single goroutine drain whole channel all the time
* mnemonics for channels "<-chan" read channel, and read from channel. "chan<-" write channel, write to channel
* channel made without size is unbuffered, it will block immediately whoever reads or writes to it
* leaking goroutines is goroutines are trying to write to channel that nobody is reading from. This is not checked by runtime. Need to close such channels with WaitGroup for example so that those goroutines will panic on write to or stop reading from. 
* binary semaphore is useful for running single goroutine "make(chan bool, 1)"
* counting semaphore is useful for limiting number of goroutines (so we don't exhaust OS resources): "make(chan bool, 20)"
* use sync.WaitGroup to synchronize that all channels are finished
* use sync.Once for one time initialization in concurrent environment, also called Lazy Initialization
* use sync.Mutes Lock and Unlock
* use sync.RWMutes if you have exlusive lock (for read and write) and non-exclusive read only ownership.
* Memory Synchronization. It is important to use locks correctly to help runtime and OS optimize CPU level caching for concurrent execution. Mutexes can signal to OS to flush the cache.
* use race detector tool

* go has vendor specific config for packages, so you can remap where to get packages from
* can run godoc server locally
* if package contains "internal" name it can not be imported from more than current level of packages (different parent)
* circular dependencies are not allowed
* in tests to avoid circular dependencies, make new module that is imported in both modules. For tests "backdoor" private module level variables into export single test file, this file often called "export_test.go"

* go has tests (Test...), benchmarks (Benchmark...) and examples (Example) for testing
* elaborate testing frameworks are discouraged "they feel like foreign language"
* define tests very simple first, it is bad to have abstract test utilities
* white-box testing — test that internals of package are correct, not just interface.
* avoid brittle tests —tests that change often with changes in code, try to make tests such that they remain even if code changes (e.g. match substring of error message)
* go command line has tool for coverage reporting and visualization
* go benchmark is easy to use and define
* for benchmarking use different sizes of input and algorithms to see their behavior and comparative advantage
* go command line also has profiling. Profiling shows which functions use most CPU, memory, or blocking 
* examples in testing are used in godoc to show how code is used. This is nice since it is showcases how to run it and go compiler checks that code is correct. If it has "Output" comment in body, then go test will check that it produces same output to std out.

* reflection, powerful, but easy to make mistakes. better stay away from it
* unsafe allows access OS memory address. However, addresses can change during runtime, Go spec does not guarantee their stability.
* layout is not guaranteed by Go spec
* unsafe.Sizeof may be useful to check size of structs
* calling C code looks easy in Go. Calling go code in C also possible. Check cgo module.
Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1549253 2020-05-25T09:29:53Z 2020-06-08T14:58:49Z scientific-software

Let's look at serious scientific and high performance production grade software

UPD: 2020-06-08

  • SpaceX software in rockets is in C++
  • Tesla is likely to ruin C++ and Go (based on their github)

C++ is leading by a large margin. Its ecosystem is astonishing. Pretty much everything of significance has been written and maintained in C++. Bare in mind, other languages like Golang or Rust have been around for around 10 years, still main applications did not migrate to them. Even more, TensorFlow itself is not in Golang, even though they are both from Google and Tensorflow created 6 years later. It is true that Golang is having momentum in service development — things like networking, ORM, business logic — yet it is used in not much anything else.

Some languages was made by mistake (Javascript), some languages are too minimal (Golang), some are not scalable (Python). Maybe, then C++ has plenty of good besides performance? It has first class OOP, generics, operator overloading, standard library, gRPC and Apache Thirft, with RAII memory leaks are rare. Where I am going with this? If you are going to write high performance software likely it is going to be in C++.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1535740 2020-04-26T09:34:57Z 2020-04-26T09:34:58Z dust-and-sweat

It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.

-- T.Roosevelt, 1910

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1519833 2020-03-14T18:15:12Z 2020-03-14T18:15:12Z iphone-6

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1515185 2020-03-01T17:09:07Z 2020-04-12T06:52:06Z bigger-picture

Your project is not good, manager is an asshole, stock is going down, back started to pain again. Finally, you got the award you always wanted, promotion, recognition. Problems, plans, achievements — it's all you. Everything is circling around you. This is your world and you are the in the center. From day 1 and until the end, you are the lead of this story.

Yet, your are small. Your world is a succession of days. With the rising sun, your day begins. You are having a breath of fresh air on your way to work, thinking what you will say to your buddies, then talking to them, getting through the day, finishing the job, thinking of a dinner on the way back home, and, at last, going to rest, until another day begins. In reality though, sun did not go up, it has always been there, fixed. It is your world that is a just a viewpoint from a tiny spot on a spinning rock circling through a vacuum of space around a giant nuclear reactor.

WYSIATI. You are a 24/7 observer of your story. It is all that matters. And not even in years past or future, but right now. Most pressing questions are whether you are hungry or happy, now. At best, you plan for the next season or a couple of years ahead. Rarely, your reflect on distant past or future. That is just you, but there are many others with their own stories. There have been even more before you and will be evermore after. What you do now can't change stories who lived in the past. Likewise, it will be irrelevant to generations far in future. Biggest evils and saints will go to an oblivion and their names will vanish in archives. What could happen will happen, with or without you. And what does is it matter to you anyways what someone in the past of future thinks of you? You are but a spec of dust.

This does not mean you should not do cool staff. Please, do! Rather, next time immediate worries overwhelm you, hold on for a moment and think about the bigger picture.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1494346 2019-12-31T06:53:40Z 2020-04-02T12:48:38Z bye korea

Leaving Korea today. How will I remember it this time?

Quality service. Civilized people. Cheap housing. Clean streets. Late hours. Latest tech everywhere.

Things just work and damn well for the good of all the people. Things improve.

Korean is everywhere. Whatever is non-Korean is lame. Living with only English is possible, but missing-out on occasion.

Strong respect of experience. Private networks. New friends. Old friends. Giving back to the community.

Better sense of things, place, time, people, and myself.

This is not the first time I am back, and it will not be the last one. I will keep visiting and when stars align will stay here with no worry of leaving.


Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1493923 2019-12-29T13:25:31Z 2019-12-31T06:04:00Z 2019 books

This year

... discovered academic publishers — MIT, Princeton, Cambridge, Wiley — and very strong material: Normal Accidents, Streetlights and Shadows, Cornered, Causal Reasoning in Physics

... expanded on practical software engineering: The Data Warehouse Toolkit, The Clean Coder, Production Ready Microservices

... looked deep inside myself: The Ego Tunnel, The Manual, The Meditations, On Having No Head

... learned to write a bit better: Dreyers's English, Zen in the Art of Writing

... followed powerful stories: life of Wernher Von Braun in Dr.Space, willpower in White Devil's Daughters, horrors of Night, great leap of Rocket Men, cheerfulness of Jackie Chan, neuroscience in Into the Grey Zone

... got some good life advises: Never Split a Difference, Getting Things Done, Made to Stick, 12 Rules for Life, Range

... touched variety other interesting topics: What We Cannot Know, Brief Answers to Big Questions, Spying on Whales, The Art of Invisibility, Empty Planet, Novascene

... and just enjoyed some good entertainment: We are legion, Exhalation, What If, Soonish

43 books, longest 460p, average 270p.

Almost all paperback. Almost all gifted to charity, library@coex and friends.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1489280 2019-12-15T08:31:43Z 2019-12-15T08:32:59Z 2020s

I got this idea that it would be fun to make a couple of wild predictions and later to go back and see how far I was off. So here is the list,

10 years

The next big thing is going to be Brain Computer Interfaces (BCI). Data transfer speed will increase exponentially. Materials will get less invasive. Price will drop exponentially. Of course, at first there will be roadblocks, but once first great leap is reached — reading at 2x, memories download, skills upload — private and government capital will pour in. First trials will quickly finish in early 2020s. The race will be at full speed in 2027

Outer space is unbelievably rich in commodities. Yet, keeping human-friendly conditions out there would be even more expensive. All operations, especially in deep space, will be fully unmanned. Software will support whole life-cycle — reaching remote places, extracting materials, building facilities, transporting them back, conducting scientific experiments. Technology is almost ready. Mars and asteroids are pretty far to reach, but moon is very close. First economically meaningful Moon base will operate by 2028.

All main languages — Python, C++, C#, Java, Javascript — will still be around. SQL and DBs that power it will gradually absorb best practices from all the variations that will continue to emerge every year or two. Perhaps, more focus will be on persisting and sharing computed (in-memory) state between processes to save time on restarts and deliver updates faster. With high network speed, there might not be a need for cold-storage on user devices. The big thing in computing will be doing global state very well or ways to operate equally well without it.

Social networks will be controlled by governments. Mandates will be imposed on what you search and how you talk. Access to mass spread of information or social organization will be tightly controlled by governments. USA will likely to protect individual privacy at least in some form, meanwhile places like China will require complete transparency to government even in private matters. On the bright side, similarly to what Instagram did to unlock ephemeral social value, new kinds of apps will emerge. Instagram itself will be replaced by its successor or transformed beyond recognition. Emphasis will be on real physical connection and local communities and businesses. In the end of the decade, VR and BCI will be the new place for social interaction. Hopefully, email will still be around in 2030.

People will continue to travel a lot and keep learning from different places. This may not make public to act coherently, but will influence its choices, such as what accounts for good health care or not, what is good security or not, what is good transport or not. In shadows, public will evolve — one by one, individuals will get smarter and with that, their aggregate, public, will get smarter too. Large waves of migration could happen. Institutions will get stronger too. Starting at big-tech, wave of growth mindset will spread to every big organization, the ones who adopt it will prosper. Many governments and nations will go through self-reflection as well, hopefully leading to better life for everybody. Developed economies of US, Japan, China, Korea will grow primarily driven by dominant position in certain niche of high tech sector.

20 years

It is hard to envision what it would be, but it is possible that new kind of computing stack, from assembly up to the high level abstractions, would emerge, which is not based on Turing state machine. It will be faster and more robust for distributed computations that heavily use network. Lowest level software stack will be powered by a system only remotely resembling assembly.

At the end of 2040s, AGI is achieved, but singularity does not happen. There will be emergence of fully digital cognition that is similar to humans, but its exploding self-improvement will not happen for quite a while due some fundamental physical or mathematical challenges we can not fathom today. It is also possible that this already happened in late 2010s, but we were not aware of it. You will talk to new digital species over your computer in 2038.

In most exotic form, BCI, VR, AGI, industrial automation and space programs — all merge together into one symbiotic platform. But then, maybe, none of this will really happen. Time will tell.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1481552 2019-11-24T12:36:21Z 2019-11-24T12:36:21Z 3D

Look around, do you know where you are? Maybe you are in your room, maybe you are in a cafe, in a car, or outdoors. You must be clearly convinced you are in 3D space filled with all sorts of 3D objects. This sense is fundamental to your reality. But how do you know all that?

All you see around is colorful 2D shapes. And yet, when you slightly move, these shapes start to deform and overlap. The closer you are the bigger the changes, the easier it is to spot them, the stronger is sense of spatial awareness. There further you are, the less objects change, the harder it is to judge their positions and shapes. At horizon everything becomes 2D. Then there are heuristics that make whole job a lot easier — gradients that mimic shadows, dark and bright colors, continuity of colors and shapes, rules of projections. Funny enough, they don't work all the time leading to abundance of illusions and creatures that exploit these illusions for survival. In the worst case, if you see a car far away — no parallax, small shapes, no gradients — there is no basis to believe it has 3D shape you believe it has. For all you know, you have seen cars before, which leads you to believe it has that particular kind of shape. Next time try it yourself to observe your thought process when guessing shape of object away, you would get just a recollection from memory. Maybe you wold even do some reasoning, if that thing really got your attention. If it is something unseen you would get absolutely no idea. This is basically Tesla Autopilot.

It is easy to think of evolutionary interpretation. Why do we need 3D? It allows more accurate predictions on how world changes. It would help dexterity at your tasks for food gathering, hunting, moving. Why we are so bad at 3D in distance? If we measure complexity of world by number of objects and number of their interactions — nodes and edges in world graph — then 3D world quickly outpaces 2D version with size. Besides, the most important things to you are probably nearby — food, danger, friends. If something is far, then it is pretty safe to get minimal understanding about it until you get closer to it — or it will get closer to you! — only then you may think how it fits into your 3D world. Simple queries for objects far away such as "which direction it moves?", "what it's size?", "is it a single thing?", "what kind of a thing?" all work just fine with fast and memory efficient 2D memory and reasoning. Why do we need heuristics then? Even for small sizes, 3D processing may requite a lot of effort. Meanwhile, due to short distances it has to be fast and reliable. Thus, it is a good idea to have a bunch fast loosely coupled heuristics. It is also interesting how other creatures developed their perception of space due to specifics of their environment. Worlds of bugs in dynamic micro-scale, large creatures in rain-forest, whales in wide oceans, and birds spending fair portion of their time looking to flat surface with all sorts of things — all must be very different from ours. Just as interesting is how subjective perception of 3D changes with echolocation or sense of electromagnetic fields — is it similar to ours, fake 3D on top of 2D imagery, or is it a totally new sense like temperature?

In the end, it is just mind-blowing that 3D world you live in is nothing but a 2D image and a sense. This sense is not even absolute, but rather a continuum from very strong nearby to weak far away. All heavy lifting is done behind the scenes by evolutionary algorithm encoded in neural cortex. It is working hard to provide you with most accurate and fast representation based on all sorts of clues and extra signals. You don't even notice it. When it fails, you don't realize it. But most of the time it works spectacularly. You wholeheartedly believe it and start live in a world of its output even if it does not directly exist anywhere. Truly marvelous tech right here.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1428414 2019-07-06T12:34:17Z 2019-07-06T12:44:41Z 1969

You are working on Apollo 11 mission,

> in 1 year pocket calculator, in 3 years version control, in 10 years UNIX, Bash, SQL, Excel, in 20 years Python

... you do computations by hand, you keep hand written records and charts, you code highly resilient asynchronous systems in assembly without version control

> in 3 years knowledge base, in 20 years website and arXiv, in 30 years Wikipedia, in 40 years StackOverflow

... to get the information you need, you read the printed manuscripts or find someone who read them before you

> in 4 years mobile phone, Ethernet, email, in 20 years IRC, in 40 years Slack

... you share data in printouts by fax, you do all meetings in person or by landline

It worked. We put three man on the Moon and get them back, followed by nine more — the goal so ambitious it has never been accomplished again. What do we do with all the marvelous tech available today?

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1392895 2019-04-02T15:36:28Z 2019-05-18T02:35:26Z foreign language
There is something special about medium where you don’t speak language. As you move in, a brief friction is followed by quiet. At its busiest, you remain in your personal space. Your thoughts can reach further, stay sharper.

Ads don’t work on you anymore—you don’t get distracted on them as much. It is surprising how much power they loose without appropriate textual or verbal message. Certainly, vivid imagery still provoke emotions, but, without catchy message, now it is up to you to come up with interpretation. Ads are notoriously crafted to stick in your mind and alter the way you think. They don’t make you smarter. If anything, they plant into your head agenda from people whose intentions go hardly beyond getting your money, by any means. The dirtier the intentions—the stronger ads become. But now, that you break free, wouldn’t you want extra capacity to focus on what you think is really important?

You don’t get involved into surrounding chatter. Of course, it is good to peek sometimes into what other people are up to, but it is your choice now whether to focus on that. Their life gets private and you don’t get any extra cognitive or emotional load. You get an On–Off switch; you get a good sense about people from the way they talk; you observe better; non-verbal signals bolden. And you certainly get the same feeling from the place. Busy Philz in San Francisco, BlueBottle in New York, CoffeeNero in London, CoffeeBean in Seoul—all keep their ambient charm. You are going to get the same vibes wherever you go.

But wouldn't be communication hard? No, it's rather the opposite—it becomes more efficient. Most people around you don’t speak English well, although, given enough opportunity, they understand it perfectly. You start talking more clearly, carefully choosing the right words to convey the exact message in shortest way possible. You strip it to the essence. People who do not get used to this mode of communication try to tell way more than they actually need to, it overwhelms and leads to frustration. You give people more space and become better listener. You may not eloquently converse about ingredients of some dish at restaurant, but you would also never face anything unresolvable. After all, people still know what thumbs-up means. It takes only one good word at the right time to make the day brighter. Make it count.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1376157 2019-02-20T06:41:00Z 2019-07-06T10:13:44Z control your data

1. Back-up your social data. Download everything from your networks and get a private paid blog instead. Be in charge on what happens to your posts and connections. You don't want to be one day locked out of your social circle due to some algorithm blocking your account with no way getting it back. 

2. Use channels that you have control over. Don't let someone nudge you on whom you should connect to and how. Companies optimize for their own objectives, make sure their true agenda aligns with yours. Better yet, control channels yourself. Some good channels are Twitter, Zoom, Signal or even vanilla email.

3. Get control over your news channel, get RSS or Feedly. Choose how you get news yourself, be that in chronological or other way. Choose sources yourself. If you get suggestions, keep the only source of suggestion that you decide is worthy. Control what you see, how you see and when you see.

4. Keep a single cloud service, purge everything else. Use Apple Image Capture to purge Photo library on iPhone. It is easier to manage and reduces attack surface. Use cloud service that allows you to download and delete all your data back in bulk, some providers force you to do it file-by-file. Don't upload to cloud raw data, pack it and encrypt it first by yourself. Don't use cloud as a transport between devices, connect devices directly, it is fast and secure. One day cloud provider can lock you out, get offline back-up. External SSD or HDD is perfect. Apple Photos works well with stand-alone offline libraries and has all the regular nice features. You can also use shared albums without uploading everything. Back-up photos, videos, music and iPhone settings separately, it makes back-up fast and consistent. Apple Photos Import has nice option for "delete after import". Remove duplicated photos and the ones you don't care about, the less data you have to carry around the better. Don't do hoarding of things "you like" or "will check in the future". Chances are, you will not check that at all, it wastes your time and space.

5. Harden your OS.

6. Use email client that respects privacy and doesn't serve ads. Apple's Mail is good option. It has rules and compact interface that is one-to-one match to gmail, before they switched to Inbox. Turn it on at "Mail > Preferences > Viewing > classic layout".

7. Get to know "private mode", Privacy Badger, HTTPS Everywhere, uBlockOrigin. In some cases, you might want to consider Tor and Tails.

8. Get MFA for key services. Get Google Authenticator in case you change phone number. And get YubiKey in case you lose your phone altogether.

9. Use only apps you trust. Check periodically what they are doing in Activity Monitor or even better - Little Snitch. If your music streaming app is downloading and sending tens of MBs per hour of data even if you don't get new songs, there is something wrong with it. Get to know VLC, Transmission, Firefox.

10. If you use CLI, know what you are running. Open source does not automatically mean secure. Use your own judgement™️.

11. Get self-hosted VPN. Make it a default when you connect to the internet. With few exceptions, almost nearly all web is accessible just fine with it. Your network speed may actually improve.

UPD: 2019-07-06

12. Disable loadable content in your email client. 3rd parties can track time, email client (and thus platform), IP address, geo information, fingerprint from your emails. And whether you opened email and how many times.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1369841 2019-02-03T10:24:34Z 2019-05-18T03:11:27Z transport

To begin with, if you driving a car in urban area, you are contributing to traffic congestion. Cars are highly inefficient, they occupy uneven space per person. Think of how many people can be packed in a bus in a rush hour? Twenty? Thirty? What if each of them had four-five meters long metal box? Add to this two meters of gaps between the cars. Why they are taking all that space? Cars don’t accelerate immediately, rather there is a delay spreading in a chain from car to car. Even worse in road junctions – time to change roads comes in fixed time turns. After four minutes it can be either forty people changing their street to continue their journey, or a single one. Why their time is more valuable? We all pay for that nice expensive road you are using as your own to place your personal, more comfortable, space on a wheels, it is only fair if you pay for it appropriately, and not to oligarchs of car-making monopolies, but to people right in front of you, sitting in a bus or an ambulance, whom you delay.

It is weird how not-weird cars are. Did you notice that all cars looks the same? How many cars have you seen with two lights on the sides and one in the middle? How many have the whole horizontal space with lights? How many lights have triangle shape? Why do you always see lightbulbs inside? It does not matter the size, price or type of a vehicle. They all look like some kind of animals. That design is intentional – if people see something big alien moving fast towards them, they will freak out. Did you notice that most of the cars have their windows tinted? You can’t see a person behind a wheel, this completes the image of the car in your mind as a single entity. The cultural footprint of this phenomena is so strong that it is not weird at all for adults and kids alike to enjoy countless movies about cars getting their own will or obtaining humanoid forms. Just look at what advertisers anchor cars with: bulls, horses, jaguars. In reality though, there is always some adult sitting inside, trying to relax from not hitting anybody in search of parking, all while being totally lost from the fact that he is inside of an overpriced metal box that looks like some futuristic tiger.

Now, putting these pricey fetishes aside, let’s consider one important implication – once you behind a tinted window, you feel separated from what is going on outside. After all, nobody can see or hear you. You are becoming anonymous, and, as with any anonymous environment, people becoming the worst of themselves. You can be as big of a jerk as you like when nobody can see you or hold you accountable. This becomes a problem. Think about a pedestrian crossing: these are protected by law designated areas for pedestrians to have priority of crossing the road. They are designed for convenience and safety of walking people. Now, imagine you are crossing the street but all of a sudden there is a huge metal box – in shape of futuristic tiger – rushing at you at 30+ km per hour; flashing bright light; loudly honk. If it hits you, you get damaged pretty badly, maybe even permanently. Your future life is at risk. But for person inside though – unlikely, besides prosecution, if somebody finds him. And even then they may get away with minimal penalties, perhaps taking away their driving license, temporarily. Your unconscious knows this too, and after it does the math, you are likely to stop and let the car pass. Now, the driver can clearly see the pedestrian crossing and knows that you should come first, yet he can just speed-up and get away with it. Nobody can honk back. Nobody will see your face or take a photo of your driver license. Even if they did, what they gonna do with it? I can only imagine what sick feedback loop it reinforces in a driver’s mind. Personal cars is a risk and accountability transfer machines poisoning city life. So what can you do?

There is one common form of action I see people taking – intentionally ignore drivers; look at the smartphone instead of the road. Don’t do that, it is dangerous and makes you look reckless. For once, whenever you feel safe on pedestrian crossing you can just go first and make car stop. There are many factors that will make it scary for you, but try to reason with logic. Cars do regular inspections and are constructed to stop quickly. Nearby to pedestrian crossings there is always maximum speed limit that is just enough for vehicle to stop, unless they are speeding. In most countries, pedestrians have priority protected by law. As long as they are a bit far and can see you – go. And while passing by, make sure to look at the driver, not the front of the car, to place where driver sits – “I see you what you did here”. For evolutionary reasons, human perception got very sensitive to anything that looks at you. However brief, even one-way eye contact breaks sense of anonymity and wakes up the conscious. One word of caution, watch-out cars that are accelerating instead of stopping. There are certain drivers who lost their mind. Try to report them to police. Even simple email can be of use. That person can be just as insane away of driving wheel as while using it. Maybe, one day your report will help investigation to fill the gaps in a more serious case. Of course, all above works if people have respect for the rule of law. If you leave in U.S., maybe you should stay away from the roads. If a person at the wheel is mentally ill or criminal with a gun, meaning they are not afraid of law due to their conditions, then, perhaps, it is wiser to stay away from that car. If that is common in place where you live, then my question for you – what are you still doing there?

But is there a better way to move around? Think about pooling money with other people to buy one big vehicle, so that you can ride with them and utilize public transport infrastructure efficiently. You setup good coverage so that you don’t have to walk too far or wait too long for vehicle to arrive. You appoint dedicated people to drive them all the time, so none of you have to worry about parking on maintenance. Due to high volume you can utilize economy of scale, meaning parts, fuel and associated costs are minimized. Sounds crazy, I know, but hold on, you can also make government to subsidize it and take care of all the operations. One step further, you move roads themselves underground, so that people never have to cross the roads and vehicles have never to wait, even in the rush hour in downtown! And why not making all of them electric by default? I am sure someone in Silicon Valley will re-invent that at some point. The net amount of monetary and time waste that comes with private transport in urban areas is incredible: overpriced vehicles; maintenance of vehicles and infrastructure; fuel. I would not be surprised if only excess of fuel was enough to power extra space program to the Moon or even Mars once a year. However, effectiveness of public transport holds on strong culture of sharing: that people are okay with sharing their space with others, at least for a short-while; that they can be civil and respectful to each other for 40 min; that they feel safe and comfortable in public spaces. Technology is not a bottleneck, it permits good transport for many decades already. You don’t need an AI for that, just a certain level of public trust and manners. This certainly does not hold in U.S., place of the worst public transport to date.

Who is to blame? The U.S. government, who is a corporate sell-out, and public, who can’t find a backbone to fight back. It gives me a bitter feeling to see San Francisco trams used primarily as a museum exhibit in sightseeing tour around this beautiful city. Similar case is in London, where Underground and, to lesser extent, two-deck busses are both museum and transport at the same time. Only example of transport that is working in the Bay Area is CalTrain – old train that goes once an hour (in Asian metropolis that would be once in 10 min) and has single line (not even a network). Sometimes, it is just as packed as trains in famous rush hour in Tokyo in the old days. In the Bay Area pro-car lobbying and trusts killed wide transportation network. If it was build, there would not be a housing crisis. Rather, new urban centers would have emerged and people would be traveling all around the Bay for work and leisure any time. Think of vibrant nightlife of London, but on the hills of San Francisco. It makes one wonder how different Bay Area would look like and what kinds of innovation people would come up here. If you live here today, you live in a strange version of the past, but I wish one day to see this area progress and bloom.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1359735 2019-01-03T10:32:14Z 2019-01-03T10:59:25Z running

In 2018 I reached 364km across 48 runs, year before that it was 200km. Why do I do that?

It is simple. It is a rare distraction-free time when you have space to collect your thoughts, reflect, plan, think. Quiet, slowly changing environment of a city and background chatter of people passing by has a soothing effect. Or, if you prefer, you can occupy yourself with something more relaxing. Narration in audio books has never been so vivid. Tunes in music have never sounded so well. Things happen all the time. With all the busyness, leave an hour a week for yourself.

If you run in a new place, it is also a good opportunity to connect to the place around you. What it is like in early morning or late night? Life is busy. When you grow up, apart of shopping or commute to work, there aren’t many reasons left to explore area where you live. Running gives you this occasion to see what the place is really like beyond your house or insides of a bus. It is refreshing, once in a while, to step-out and look around.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1356572 2018-12-24T06:48:18Z 2018-12-24T16:14:05Z books of 2018

Another great year. Finished 34 out of 22 planned books. Read a couple I had in back of my head for while and made few great discoveries along the way.

> What is your top pick, Nikolay?
> This year, I would recommend everybody “It does not have to be crazy at work”.

Powerful. Chief talent officer from Netflix shares fresh look at what is important to people at tech company. Recently, it become a cliche that tech company has fancy campuses with chocolate fountains, free food and champagne. Unnecessary perks, weird nudges and intrusion into work and after-work life is a wasteful corporate enterprise that diminishes agency of people working at the company. Instead, companies should focus on what they do the best — plain work. It is time for companies to cut the bullshit, think about what really matters and take people seriously. It was also great to follow story of Netflix from the front row.

Packing to Mars. Going to space is the most grandiose journey anyone can ever embark upon. But what it actually feels like? This book is full of funny and sometimes gross recollections from real witnesses. Here, on Earth, we are humans, and out there, in outer space, we are humans too.

Brotopia. One day, at popular computer science lecture at my alma mater, I noticed that there are 100+ guys and only 2 girls. The sheer scale of imbalance hit me. No matter what is your perception of expected imbalance this is way too extreme. To get some background on what lead to this, I looked into historical account on gender imbalance in U.S. in this field provided by this book.

The lean startup. Famous philosophy of rapid experimentation and growth for building new products and companies. Keep hand on a pulse of feedback from customers. Use tight build-measure-learn loop as an engine of your growth.

A study in scarlet. Beautiful novel from the world of Sherlock Holmes. audiobook 

The ethics of influence. Started this book a while ago and finally finished it this year. Now that we know about nudges, what is the moral grounds of government, or other authority like big corporation, using nudges for their own means? For example, when you walk into a food hall, what products you do see first and which require extra effort? Is it a healthy, ecologically friendly, or domestic option of the same thing? Do you even need it? Key takeaway, if we have choices anyways then why don't someone optimise choice-structure for social good? Another learning is that there are different types of nudges, some are quite subtle. Knowledge on how they work may not grant you immunity, yet the hope is it will tip the scales of power back into your favour.

Skin in the Game. Taleb covers here many interesting mechanisms that arise in case of uncertainty. One is responsibility and how it shapes social structures. Imagine you are seeing a doctor, do his incentives align with yours? Another one that stands out is difference of risk-sharing and risk-transferring. How come some customs and traditions spread and survive and others fade away. Lastly, if you don’t have real stake at what you have to say, perhaps we should not listen to you. And on the other hand, if you survived so far by making your own decisions, then there is some value to your opinion. Of course, that is very crude way to put it. As always, Taleb does extraordinary job articulating ideas, unconventional and profound.

Dreams from my father. Stories from formative years of Barack Obama, written by himself. Here I could follow his story with more detail and no restrictive format of political campaign speeches. The story of his childhood, race, African and American communities, in short his “audacity of hope”. audiobook narrated by Barack himself

Nocturnes. A couple of elegant and subtle novels about nightfall and music. paperback from SF SOMA 🌉

Competing against luck. Keep meaningful metrics and keep shipping. Pretty much same ideas as you would find somewhere else, Lean Startup for example, but less elaborate and in non-remarkable style. This book was a miss this year. audiobook

Structure of Scientific Revolutions. Monumental work on philosophy of science. If you are unfamiliar, in short, it is concerned with what is scientific knowledge and how it evolves. Now I realise, I got incomplete and at times wrong picture on this work when I was studying this topic back in undergrad. This work is very elaborate and acknowledges commonly referenced shortcomings, such as continuity. It was also great to learn how he blends research community and institutions that develop scientific knowledge into his framework. 

Three body problem. Famous science fiction from China. Very entertaining read.

The dark forest. Second part of “Three body problem” series. All the sci-fiction concepts you have heard mixed-up together. Story gradually losses its coherence leading to a complete mess in the end. The very ending was quite witty, but it takes unreasonably long to get there.

Machine, Platform, Crowd. Outlook on trends in tech industry. If you work in tech, you can probably add to this book, but unlikely other way around. audiobook 

Steering the craft. A crash course on writing and story telling. One thing that pops-out right away, keep in mind a persona who is telling the story. There are multiple kinds of voices that you can choose: omniscient observer, non-omniscient 3rd person, witness of story, main character, and others. This helps you to move reader closer to characters in your story. There is also a collection of other useful practical tips.

Bad Blood. Apparently, you could mislead investors, public and government for almost a decade, got a billions in founding without working product and cause physical harm to people. Thrilling story.

Principles. Life and work advices from founder of notorious Bridgewater hedge found. I am not found of the style he has chosen for this book, but looking back more of then not I agree with his advices. paperback

Gentleman. Short and intense novel about a guy who mistakenly sold his wife to devil. Narration gives you vivid pictures, almost like you are watching a play in theatre. paperback from SF SOMA 🌉

Bladerunner. Classic. I was surprised that movies completely missed a couple of elaborate dimensions to this story. That being said, both movies, and book have strong plot and convey unique and rich futuristic mood. paperback from SF SOMA 🌉

All the light we cannot see. Story about german boy and french girl in WWII. Very interesting style. paperback from SF SOMA 🌉

Factfullness. There are many metrics backed by solid facts that world is much better now than before and much better than you are thinking it is. Similarly, there are challenges and problems out there, but they are not exactly what you think they are. Why is it the case? Sensationalism in media and fundamental mechanics of how we learn about world is to blame. It was also great to follow his numerous stories from times he worked as data scientist in healthcare and government. Again, style was not the best, but most of the points make sense. paperback. many thanks to Hoa

All systems red. Good science fiction about life of an android. audiobook

Managing oneself. Rather depressing outlook. Didn’t learn much new from this book. Another miss.

Designing Data-Intensive Applications. Incredible book, very elaborate and coherent. Learned a lot about modern data systems and refined bits I already knew. You might enjoy it a lot if you work in software or just enjoying stretching your mind.

Forever War. What if there was a war that never ended?

On language. One of the most eloquent writings I have seen. Very deep material on fundamentals of language. Interview with Chomsky about politics gives fresh contrarian viewpoint on United States in past century. Especially good if you want to learn how did U.S. get where it is now.

Hagakure: The book of the samurai. Thoughts on life, economics, philosophy and other important affairs of people in ancient Japan.

Cat’s Cradle. Entertaining science fiction from Kurt Vonnegut. audiobook.

The obstacle is the way. Challenges that we face can be milestones that help us progress. Pushing this thought to the extreme, the problems we are solving form the way itself. Here author gives a collection of advices that can help you to build such mindset. Many appropriate references from ancient and recent history. Accompanying bonus interview with author was a pleasant surprise.

All quiet on the western front. Detailed and very personal account on WWI. Today is the most peaceful time in human history, yet we have not been so far in touch with reality as ever before. We see war exclusively through lens of sensationalist news camera, violent movies and games or state propaganda. Our perception of war is wicked. What is war really like? This book gives a glimpse of horrors and despair.

It does not have to be crazy at work. Very short and to the point account on work style. 

AI superpowers. I did not agree with many points Kai Fu Lee is making here about tech nor with his general defeatist attitude towards AI and tech. His paternalistic tone was annoying too. The most thought provoking and even shocking takeaway from this book is how scared of AI and its future can be someone so influential in this field.

Blitzscaling. Interesting read on how to scale-up organisations with lightning fast speed.

The curse of bigness. History of anti-trusts, monopolies and regulation. Originally I did not plan to finish it this year, but material on this topic is so well written in this book that it was a pleasure to finish it on Christmas. ❄️

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1346764 2018-11-23T14:35:26Z 2018-12-03T14:37:24Z language

Imagine you are stranded on a deserted island. You have never been on such island before. To survive you have to learn to do everything from scratch. You discover new plants and over time understand what is edible and what is not, what can heal wounds and what is good for building tools. You are keeping a diary where sometimes you leave notes on plants that you seen so far. Later, another person comes in and reads your records. Even with your help, they will have hard time getting all information about the plants. Some of it you had but did not keep. Some of you did not had in the first place because you lacked the right tools to collect it. Where does it grow? How does it change with seasons? Is there an animal who eats that plant as well? Each new plant in the diary requires an effort bringing back to life memories about it, so that you can answer as much questions as possible. It is certainly not possible to reconstruct it in full without the necessary context from you.

Now you are a robot stranded on an island. You are experimenting with plants and collecting all observations. Similarly, another robot comes in and reads your records. We are not sure how machine imagination works and what machine version of reading would mean, but imagine robot sees a plant identifier and extra information about it encoded right in its word. It sees plant's identifier and at the same time its colour, shape, density, chemical composition, relationship to other entities such animals that feed on it and types of soil it grows on. The robot can also see precise records on how this knowledge has been collected and can judge its reliability.

Today, if you read an article and see word like “business” you are left on your own to understand what it is. If you are interested only in movies, you would imagine "The Wolf of Wall Street". If you have experience in finance or have worked with data about businesses before you may know that: there are X businesses in country Y; businesses are best compared by metrics A,B, and C; each metric varies in ranges [A-A*], [B-B*] and [C-C*]; certain regions in this embedding space means T and others mean F. If so, you see information in article augmented by your prior knowledge. Now, this would be really great if you can also perceive all of it by reading some “hyper”-word that contains that extra information, something like “business:0001100101”. What if all the words are like that? Whenever you read article you get very rich understanding of issue at hand. That would be great.

But there is a problem. Each time you read this “hyper”-word you have to carry extract information. Basically, it is kind of wiki-page with as much important details as possible being shirked into some understandable by human mind embedding. Let's assume each word on in this wiki-page is a normal world and thus page is already has the smallest size. This “hyper”-word now is “business:<raw text from wiki page here>”. Carrying this blob of text is extremely redundant and makes reading slow. But is there a way to keep the very minimal amount of information necessary for identifying object of the word, something like log2 of number of words out there? Thats's just the world itself. There is some redundancy (50% in English) in encoding, but that seems to be for historical reasons. It contains minimum amount of information necessary to identify word out of all other words in language. Turns out languages has developed to minimise amount of information you have to carry around to describe word. This way, once agent has attained necessary starting knowledge, its throughput of communication is very high.

Now, are we learning just statistics of words? No, if so we would be a monkeys typing nonsense words. It is more complex than that. Instead, even the most rare, complex sentences with loose grammar and unseen words can make total sense. To make it even weirder, we learn languages incredibly quickly. More to that, languages that are developed in disconnected groups share similar traits, are equally powerful and easy to learn. Did we develop some innate predisposition to languages or did we develop languages that can be learned quickly? It must be both. The class of languages we have is shaped by our evolutionary constraints. And yet, the power of languages makes it believe that there is value on its own that we just happen to discover. An artifact that goes far beyond scope of functions imposed by constraints and objectives at time of its initial development.

The world we are living in is the experiences of physical world summarised in the language. Once you learn it, you have access to direct experiences of others and achieve temporal awareness yourself. We can find description of external physical phenomena that is able to capture its nature well enough to accurately predict what will happen in future. There is a vast amount of laws and phenomena we can discover in math, pure language. And we are just scraping the surface. What marvels are hidden beyond our reach? The space we discovered so far is as tiny as pale blue dot we call our home in cosmic scale of universe, and language is a highway entrance to it.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1327374 2018-09-30T16:06:48Z 2018-12-03T14:37:34Z instagram

I took a very good shot. My friends are going to like it! Hold on, it is really good! Here, let me apply some filter to make it even better. Let's share it to public so others can see it too! But what about all that alcohol, junk food, bad tech, overpriced items, and Russian ads all these people will see?

There is something fundamentally wrong with this social contract.

It strike me today. I have been complaining about ads and social platform for a while. I thought it is enough if I use ad-blocker and will not see them. I also can limit time spent and how I use them. But it is not just me. Whenever I post something I lead others to these platforms too, and there is a good chance that cherished story from my life becomes an intro to a dose of brainwashing sponsored by Heineken or CocaCola.

Yet, passively consuming content on platform is not enough either. I regularly check on my friends' photos, stories, and they can see that. In no time they develop natural expectation that once they post it I will come to check on it. By merely being there I effectively incentivise them to be part of the platform too. Now, even value of such feeble activity compound to activity of your friends. It is easy to miss this externality while discussing ads and individual's agency, but it is a core of the whole business model.

To be fair, twitter and snapchat are doing the same. Instagram, however, has pushed it to the extreme. I only wish to see more companies were brave enough to adopt business model of postheaven (creator pays for the service) or medium (premium users cover costs for everybody) or even traditional subscription.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1316507 2018-08-30T08:50:05Z 2018-11-25T10:49:17Z dota

Two teams of five players select heroes from pool of 110+. Each hero is unique selection of ~20 skills and properties that define it’s durability, resistance, growth speed, etc. Each can carry 6 out of 150+ unique items at a time. The game is 50min long real-time match in big map. Sounds like a lot of possible combinations, why is that then all Dota games look so similar?

All 110+ heroes can be reduced to 5 representative clusters. For example, there are ‘damage dealers’ who can deal a lot of damage in short amount of time under the right conditions (and most of the time they are weak), similarly there are ones who can easily escape (but deal low amount of damage). Higher value in one dimension comes at a cost in others, this balance is done by design. The ways how they deal damage (attacks vs spells) or escape (invisibility, fast movement, teleportation) is quite restrictive. Similarly, all possible spells can be clustered into few groups: protection, damage, obstruction or amplification. Fundamentally, differences in specifics of heroes account only for slight variations from the average of the group they belong to. Items are quite restrictive too. For each bracket of capital only certain set of items can be purchased, e.g. 20 cheap, 20 mid-range, 20 expensive. Moreover, for each category of heroes only selected few items make sense, other bring almost zero value. Generally, items do not augment skills, rather they improve other parameters of heroes -  compensating their weak sides or improving strong ones. Items between heroes do not compose either, items of player A affect player B only via second order effects, such as total damage output of the team or mobility. Thus, items do not increase complexity of the game drastically neither.

Contrary to what it looks on the surface, huge variety per dimension does not lead to huge variety of winning strategies. Specifics aside, each game, even between professional teams, follows predetermined well-defined path. But wouldn’t AI discover something unexpected that works well? Probably no, at least not in a usual sense; it is discouraged by the game design itself. There have been few surprising combos such as fountain hooks at TI3, however one-by-one they have been rooted out of Dota ever since the begging. The game is always under constant scrutiny by millions of players online as well as dev team. Besides of few well known strategies anything would lead to unavoidable gradual failure. Ultimately, the team who has the best draft and execution wins. Thus, it is not surprising that OpenAI Five is undefeatable when allowed to do hero selection as well. On a high level, Dota is closer to tic-tac-toe played in simulation of real world than less visible but complex games of chess and go.

Yet, OpenAI Five is a great step forward, but some refinement is required on what makes it so. Dota does have straightforward structure, however it has to be discovered first, ideally without any supervision. Five has showed remarkable understanding of many fundamental components of the game such as farming, scouting and keeping courier safe. Most importantly, it showed great skill on composing these fundamentals into impressive coherent long-term play. There are still minor issues with long-term games, but they looks more like to-be-fixed bugs rather than underlying issues with model or training. Perhaps, the only thing left is to learn how to train it faster.

Lastly, Dota is very tangible. As opposed to go or chess, you don’t need to be an expert to appreciate some meaningful fundamental block of actions or behaviour discovered by an agent. Advancement of OpenAI Five is very illustrative on how far our capabilities in AI are today.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1315543 2018-08-27T12:34:27Z 2018-08-27T12:34:53Z p(t|f)
Recently, while talking about books I have said

“..not everything that is not technical is fiction ..and not everything that is not fiction is technical”

In a moment later, I recollected that these two statements are logically equivalent. Yet, I sensed urge to add second statement to conversation, as if it was different. I have got interested in mechanism behind it and under what conditions it is useful.

Let’s formalise it a bit with t (technical) and f (fiction). Logically both statements are the same
~t && ~f = ~f && ~t

How about conditional probability?

‘probability of not technical if it is not fiction’
p(~t|~f) = p(~t && ~f) / p(~f)

‘probability of not fiction if it is not technical’
p(~f|~t) = p(~f && ~t) / p(~t)

They are not equivalent anymore.

It seems that I had strong inclination to treat both statements as conditional probabilities. Naturally, my next question was “is it only me”? Apparently, people often think of conditional probability when they use material implication in natural language (paper). I expect there are all sorts of gaps between formal logic and our intuition.

When is this useful?

If we posses knowledge about all the books out there then we can check if there are any books that are not technical and not fiction. However, in more realistic setting we don’t know everything at hand, instead we learn gradually about the world via experiments. Moreover, these experiments are costly. For example, if each experiment takes 1h then only 24 can be done daily. With opportunity cost it becomes much worse, each experiment takes slot from all other possible experiments that could have been conducted.

Simple frequentist approach for such tests would be count number of items that are both not technical and not fiction. We check book one by one and see if it is fiction or technical. If we have found at least some, then good — we have proven that such cases exists. If we did not found it, then we can not say anything for sure.

Now, if we know that some pool of books has higher chance of being technical then by knowing conditional probabilities we can prioritise accordingly and find a book of 3rd kind faster, simply because our test are now more effective. If our resources for tests are capped, then it becomes a question whether we can find an answer at all.

Our brain is very nuanced, but there are reasons to believe that it is Bayesian(ish), at least sometimes.

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1339640 2017-12-29T03:00:00Z 2018-11-03T09:17:26Z books of 2017
This year I have finished 16 books. This is 4 books less than year ago, but it matches the plan for this year! 🙌

It will be marked by good books I have known for a while as well as great new discoveries. The subjects vary greatly, but few common reoccurring themes can be seen.

One more observation, many of the books are related to what I have read before. Such, I have read books from Tim Wu and Michael Lewis. Later wrote the book about the author of “Thinking, Fast and Slow” that I read last year. Speaking of fiction, plus-one goes to Charles Dickens and Neil Gaiman (looks like it will be the last one for Neil tho).

This year it has been 5 fiction (2 sci-fiction) books. Remarkably, EGB is the longest I have read so far with over 800 pages. More numbers from goodreads:

The Attention Merchants Tim Wu. Just as in his other work — “The Master Switch: The Rise and Fall of Information Empires” — here author gives a close historical look on the evolution of attention industry. Things at which we pay attention influence how we think and how we make our choices. Hence, it has been subject of profound efforts to direct it by individuals, governments, and businesses. The book concluded with remarks on how to this day attention in many forms is still the principal force that shapes our society. The real story was not as lush as Mad Men, still very intriguing to follow.

The Chimes” Charles Dickens. English classic. It was nice to spice it up a little last winter with a bit of Christmas magic. — audiobook

The Nest” Cynthia D’Aprix Sweeney. It was a joy to read this light story about one big American family. I remember when I bought this book, lady at the checkout told me “..this is her first book. How did she do it so well?..”. I don’t know the answer, but I totally agree. — paperback from bookstore in SOMA SF 🌉

We Are the Change We Seek: The Speeches of Barack Obama”. Last speech of Michelle Obama as FLOTUS sparkled in me interest into public speeches. Next thing I remember I was watching speeches by Barack Obama. I have got inspired by the style, how he elaborated on his ideas and connected them. Written form also helped to zoom out and see how he constructed his speeches more clearly. — paperback from bookstore in SOMA SF 🌉

The Life-Changing Magic of Tidying Up: The Japanese Art of Decluttering and Organizing” Marie Kondō. It was truly a life-changing book for me in many ways. My home have got cleaner. I’ve got good habits, more free time, better sleep. Most importantly, this book gave me a cozy and warm feeling when it was very much needed.

The Hard Thing About Hard Things: Building a Business When There Are No Easy Answers” Ben Horowitz. This is a cornerstone of philosophy of execution under stress. This is a guide on how to go through struggle. How not to die. How to win when no-one cares.

Zero to One: Notes on Startups, or How to Build the Future” Peter Thiel. This book explains fundamentals on creating a new venture. However, to me its biggest value comes from its inspirational yet intelligent message. I mean, you get this strong sense of startup philosophy while reading this book. I am also very found of its open and direct style.

Homo Deus: A Brief History of Tomorrow” Yuval Noah Harari. First, author went through forces that shaped our society up until this moment. Next, he explored on how these fundamental learnings might shape our nearest future. He very carefully and coherently covers hot topics including: AGI, religion, ethics, politics. Among many other topics with elaborate examples, from top of the mind, I most vividly remember references to “inter-subjective” — realm where constructs make sense only when there are humans who believe in them, e.g. money, bank, government, corporation. However, there are many more ideas that are too hard to squeeze into one paragraph. Some of them are quite sobering, but sound reasonable nonetheless.

A Nation At Risk: The Imperative For Educational Reform”. Published in 1983, this official report gives harsh but critical overview of educational system in the United States at that time. The key alarming message of this report is that educational system was not keeping up with other countries. It provided analysis on why that might be so and how to fix that. I was interested in this work primarily in the context of other studies I was reading at that time.

Gödel, Escher, Bach: An Eternal Golden Braid” Douglas R. Hofstadter. This book has been a separate challenge to me. So far it has been the most challenging book I have read, but it has been the most rewarding at the same time. It is an elegant journey into math, consciousness, music, biology and art. — paperback. many thanks to Pooja.

Chaos Monkeys: Obscene Fortune and Random Failure in Silicon Valley” Antonio Garcia Martinez. This story sheds a light on the dark and ugly side of startups and big tech companies. Some of the points I find quite relatable, some — not so much. Often the story is told in very subjective and overly one-sided manner. Nonetheless, I appreciate the author for speaking-out and uncovering unspoken hard topics from the tech world.

The Restaurant at the End of the Universe” Douglas Adams. Good sci-fiction in the best spirits of “Hitchhiker’s guide to the galaxy”. I wanted to know how the story unrolled next. Had a few good laughs while reading it.

The Undoing Project: A Friendship That Changed Our Minds” Michael Lewis. This book does not talk much about behavioral science, rather it tells the story of Danny Kahneman and Amos Tversky on their way to form its foundations. This is a story of people behind “Thinking, Fast and Slow”.

Man’s Search for Meaning” Viktor E. Frankl. Long time ago back at KAIST, I have heard someone mentioning Frankl’s Optimism in his argument. Particularly, it was that even under worst conditions we are able to make a choice to strive for the better. Since then I wanted to finish the original work. At first, author brings very vivid and frightening images of struggles in death camps. Later, he uses them to advocate his theory. I am not sure about its soundness, but I like that it gives more weight to top-down (slow system first) directions of our motives as opposed to bottom-up (fast system; subconscious first).

The Time Machine” H.G. Wells. Very well written short sci-fiction. Author explored ideas on how basic biological evolutionary forces will play in long term with technological progress and social order. — audiobook

Fragile Things: Short Fictions and Wonders” Neil Gaiman. Last book this year. The choice was between another classic — Sherlock Holmes, and this book. I chose this one due to good memories of reading his previous collection “Trigger Warning”. This one is a collection of mostly dark stories. Few are good, but only very few. Perhaps, Sherlock would have been a better choice.— audiobook

It has been a great read this year. Let’s keep it rolling!

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1339642 2017-05-19T03:00:00Z 2018-11-25T10:49:08Z finished KRYPT

I spent quite some time on it but enjoyed every bit solving it. Along the way I made few surprising discoveries about the puzzle that are intimately tide up to computing.

For instance, pretty soon after I have started I could see that the key problem here is how to find the right block quickly enough. Since the pieces do not have fragments of picture painted on them, the only way to identify the piece is by features of shape. Turns out, even though pieces are misleadingly similar, they could be uniquely described by width, height, type of shape (5+), shape or holes, and shape of arms. In a sense, now this is a computational problem on how to “query” (find) these objects in the most efficient manner possible. Put it this way, we could speed-up the process a little bit:

Alignment — will help to see the differences easier. Analogy in computing would be padding, for example in the low level of filesystem routines to speedup sequential access.

Group by type — no need to do search in all pieces. Search in subset makes it few times faster.

Order by same selected feature — duh.

Least Frequently Used policy — cache eviction policy that is used to speedup access when fast storage is very limited (cache). It is much easier to look and try pieces from say a group of 5 than 40. Think of those 5 really good likely to fit-in pieces as your cache and the rest as a RAM. How this works with this puzzle? Let’s say you have a group of 40 and you a searching among them often to fit a segment of similar pieces. If after each failed try to fit an item you put it in the corner, then eventually you will improve quality of your group by having very similar pieces close together, which in turn will let you query them faster in the future.

it’s all started like this

first steps
now we get center. but oh boy, everything else is such a mess
doing a bit of cleaning
well, that was helpful
now we have borders! 
more stuff emerges
more alignment, more grouping, more sorting!
yay! 

Nikolay Dubina
tag:nikolayid.posthaven.com,2013:Post/1339644 2016-12-28T03:00:00Z 2018-11-25T10:50:00Z 2016, year in books
As a part of my last year’s resolutions, on the January 1st I committed myself to finish 20 books this year. And Yay! I did it! 

Cool thing is that in this year I read more books in English than in the rest of my life before (5 in 2015, 3 in 2014 and only a couple before that). Some of them are pretty short, but on average they are of a decent length. Many of them are popular, but few are surprisingly undiscovered. Here is what goodreads tells me:

Here are few words about each of them:

The Education of a Poker Player” — in this book cryptologists Herbert Yardley, who founded US black chamber (predecessor of NSA), guides through his life experiences and teaches subtleties of a poker game.

Age of Ambition: Chasing Fortune, Truth, and Faith in the New China” — China have long been of a big interest to me. This book helped me to gain a bit of insight about its today’s society, role of party, culture with a bit of deeper historical perspective. Author recalls his memories and conversations with many people from different backgrounds in China, which forms very tranquil and rich narrative.

The Lessons of History” — this book is a collection of observations regarding biology, social order, economics, religion, moral. If we could say few things about each of them, what would they be?

Brave New World” — widely known masterpiece that became classics of modern literature.

The Vital Question: Energy, Evolution, and the Origins of Complex Life” — with innate to academic world love to detail and clarity author describes origins of life. It is certainly filled my gap in this part of biology. But what is more, I found it interesting how author brings physics and biology together and goes into details of evolutionary process.

Trigger Warning: Short Fictions and Disturbances” — collection of short, a little bit scary and disturbing stories from Neil Gaiman. These elegant stories were like a breath of fresh air in the midst of predictive, worn-out plots we see often in TV or novels.

Sphere” — very pleasant Sci-Fi in the best traditions of the genre.

High Output Management” — in this fundamental book Andrew Grove, the guy who made Intel, US semiconductor industry, and the Silicon Valley itself back in the day, talks about how to be a Great manager. We are all managers of a sort — we manage our own lives and people around us. I find it uttermost useful and insightful.

The Machine Stops” — another short Sci-Fi that pictures over-reliant on tech society.

Ghost in the Wires: My Adventures as the World’s Most Wanted Hacker” — true story from the O.G. hacker who elevated social engineering up to an unimaginable level. Reading this book was comparable to watching Hollywood Action movie that makes your heart pump faster, but with a big difference — the story here is real.

Daemon” — (if you are involved in software world — yes, it is that daemon. background process in your OS). What if highly autonomous, self-healing, distributed, and resilient program runs wild. How bad it could be? On the bigger scale, tech described in this book blurs border between highly advanced man-made tools and intelligence. Cool thing — folks from Google were consulting author about tech :)

Freedom™” — second part of Daemon. Now whole society is going through rapid changes and being re-shaped by tech. Is it good or bad? This book reminds how dramatically society can be changed by tech.

Outliers: The Story of Success” — this books is very popular nowadays, I see people reading it in London underground and in coffee shops. Without going into details, in this book author tries to find common patterns behind stories of success. No wonder this book is so popular :). It is a combination of luck, hard work and environment. What surprised me the most is role of environment and culture, which I tend to underestimate in favor of other factors.

Pale Blue Dot: A Vision of the Human Future in Space” — aesthetically beautiful passages about other worlds and outer space. Imagery that you picture while reading this book is comparable to ones from best Sci-Fi (in my case — Interstellar), but it is real and goes in hand with our best understanding of physics. Interestingly, few times you can even see glimpses of technology that might come only in decades (i.e. today) like rise of A.I. enhanced tech and VR. And all these has philosophic foundation that sees us as travelers, explorers and defines a new frontier in space.

The Last Lecture” — there is a heart-breaking story behind this book, which made me curious about its content. It is a collection of advices reflected in life story of a man that happen to be in very unfavorable conditions. There are definitely some good ideas here, but to me it was more about understanding how he thinks under such circumstances. It is one more reminder for all of us think about what really matters.

The Man in the High Castle” — what would our world looks like if history turned other way in WWII? The plot in the book goes closely to the TV Show. However, I find later better developed with richer and more interesting narrative.

Utopia” — very old book, really, published in 1516 in Latin. It is mix of fiction and philosophy about social structure and politics about imaginary society. It is where term “utopia” is originated.

Thinking, Fast and Slow” — despite it sounds like a cheep motivational book. Believe me, it is very far from that. This work is written recently by Nobel Memorial Prize in Economics laureate Daniel Kahneman. It is like a comprehensive course on psychology and cognition have been compressed into a book. Yet, it preserves good writing style. It also has many references to economics and statistics. If you are familiar with books by Nassim Taleb, you will find here many related ideas. The core concepts in this book change the way how you understand human behavior and inner workings of our mind.

Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy” —just like in “Silent Spring” by Rachel Carson, Cathy O’Neil talks about issues that require public attention, but on different front — big data usage in big companies and government.

Bushido: The Soul of Japan. A Classic Essay on Samurai Ethics” — this is another old book published in 1900. It is one of the best essays on the traditional Japanese culture and the place where Bushido was introduced to Western world. It is one of the works that initiated such prolonged interest in Japanese and Oriental culture, so I was eager have a look into it myself.

Looking forward to what I will learn in years to come!
Nikolay Dubina