Introducing Familiar, a quantified reasoning assistant (feedback sought!)

Maybe some of this is a little far fetched but god would it be fun to dig into.

My sentiment exactly! This seems like the sort of thing that would enable people to seriously improve their lives in a lot of different ways, but there are also many more ways to use it that probably wouldn't help much or at all. That's why I'm trying to focus on ease of use--the more people out there experimenting with measuring different things in combination, the sooner everyone gets to benefit from those methods and combinations of measurements that actually do help.

Just reading its documentation has been very helpful so far. Thank you!

Yes, I'm pretty sure this is currently the most significant hurdle to getting people to want to use it (this isn't the '80s!). Adding some kind of browser-based GUI is my current task.

I would immediately download this iff it had a GUI.

So immediately download it!

That's precisely the reason why I gave up and am building my own Bayesian classifier to do ... almost exactly what this post's project will do. Only, mine is meant for strictly personal use and is related to hashing out a diagnosis with the help of a doctor.

"The unquantified life is not worth living" -- pseudo-Socrates

"The value of an unquantified life cannot be determined exactly."

Hooray, people with credentials! Thank you for sharing your knowledge.

This was my most convincing reason to try to bother implementing the statistical guts myself in the first place; it was pretty easy to put together a little naive Bayes classifier that calculates the maximum likelihood estimate for all your other variables/predicates given the value of one variable/predicate intended to be minimized or maximized, and I'm pretty sure it works mostly correctly, and I'm pretty sure the additional return from using virtually any of the existing more sophisticated ML algorithms that I've yet to hear of won't be nearly as high as the initial return from being able to answer the question "if this is the case, what other stuff was most likely the case?". I'm starting to get the suspicion that the next most useful modeling-related task may be to focus on generating lots and lots of different compound predicates given all your raw variables in some way that doesn't reek of unhelpful combinatorial explosion, then calculating the maximum likelihood and probability of that value for all of them using my existing dumb little classifier, which isn't something I can recall seeing any work on. If that reminds you of something then I will desperately consume whatever resources or names of algorithms that come to your mind.

In the near term, very low-frequency events that have a very clear impact on other variables might be usefully grouped under the raw variable "significant and obvious other things that aren't worthy of their own variable in my opinion", and many of the more useful terminally-valued predicates could imaginably require that variable to take a certain value/range of values. Maybe "rejection by people currently in my social circle" with a holistic, multi-valued rating could be its own variable, if that happens unfortunately often for some people. I don't expect to automate measuring something like this any time soon, but it is undoubtedly important to know about in figuring out optimal conditions for "normal days", which makes me think the manual data entry part will be unavoidable if you want this to be particularly useful.

In the more distant future, I see no reason why such a thing couldn't be learned into a variable that actually does an okay job of carving the important parts of your personal reality at the joints. If you have a software system that knows relationships are important to people, knows which of your relationships are important to you, knows who you were talking to, and knows the valence, arousal, duration, frequency, etc. of your interaction with that person over time, then, yes, something like "ended a relationship today" probably could be inferred. It doesn't sound trivial but it sounds absolutely plausible, given sufficient effort.

All the ones I've found so far have some combination of the following traits:

• Proprietary and expensive
• Steep learning curve/assumes the user will be a technical professional
• Not designed to use data from other services

I think a program that avoids all of those things simultaneously will be able to fill a useful niche for many people. If you know of a free program that an intelligent layperson could learn to use with minimal effort, and that's made to collect data from a variety of existing Quantified Self apps and devices (or at least be easily amenable to such extension), I'll happily stop development and use that instead.

For #1, just use readline/editlne. Don't try to implement your own.

Although personally I'd prefer a nice shell-scripting interface. (Note: I have not actually tried the program or checked if it has one.)

It looks like getting editline to talk with my Clojure code would be somewhat non-trivial. Using the Clojure REPL is probably the better alternative for serious use until I get a GUI working.

Reading CSVs is now on the todo list.

It's an H2 database saved inside /data in your Familiar directory. You can make SQL queries into it with other programs. Exporting to JSON or CSV or something will happen eventually.

Two, technically, I suppose, but I'd probably collect data for a couple of months before I started seriously interpreting correlations involving variables with a resolution of one day. This will be a topic in the more extended documentation.

You want

(change-time (days -1))

if the active time is on the wrong day. The active time being wrong but on the correct day doesn't matter yet since only day-resolution variables are supported. A time zone setting will be added along with variables of arbitrary time resolution.

For percentage variables, use

(num-interval 0 100)

as the validator for a variable. (I will add "percent" as a built-in validator.)

Besides the built-in documentation for all the API functions, the readme on GitHub is the most comprehensive existing documentation. Of course I intend to fix that eventually, probably along with the next release which is going to be the one where there is a GUI.

Thank you for all your feedback!

There is, though it's not implemented as a neat API function (yet), so if you're using the official release from when this was first posted you can't do it. It looks like this:

(update variable (where {:name "cats"}) (set-fields {:unit "encountered" :fn "non-negative?"}))

This will break preexisting data if the prior validator wasn't a strict subset of the new validator. Converting variables into a new representation sanely and easily is something I plan to add in.