My BASB + LLM Wiki System

Motivation

Years ago when I was still in university, I had the opportunity to learn from Tiago Forte back when he was first getting his start in becoming a sort of note taking guru. I was slammed with trying to complete two thesis (double major), some tough math classes, an RA gig in the freshmen dorms, and trying to apply for the peace corps (under the old system mind you). I was originally just supposed to just get his advice on how to put my peace corps application together but after simply admitting that I felt completely overwhelmed and unable to put in the time to do it. Tiago calmed me down over zoom and instead offered to give me a rundown on his task management system that he was trying to sell a course on. Since then I’ve watched him go from teaching David Allen’s Get Things Done, to a class on Evernote, onto a more generalized Building a Second Brain course, and now I believe Personal Knowledge Management is what he terms his craft now.

A Simple yet Useful Local LLM Project

Hey everyone like all of you (hopefully), I too have been looking at large langauge models and trying to integrate them into my workflows in new and creative ways. In particular I’ve been enjoying working with the Ollama project which is a framework for working with locally available open source large language models, aka do chatgpt at home for free. Here’s a short script I created from Ollama’s examples that takes in a url and produces a summary of the contents. I use this along with my read it later apps to create short summary documents to store in my obsidian vault. Thereby integrating pieces of information into my boarder and searchable second brain just by providing a url, eventually I want to hook this up to the newsletters I read so I can automate the process further. Here’s the full script with added comments.

Save money use a static website generator

I created my first youtube video and it’s a overly long and meticulous tutorial on creating a completely free website with github pages, similar to the website hosting this blog.

Introduction

With the election behind us once again the electoral college, that most peculiar feature of American presidential elections is bubbling up as a part of the conversation. Typically when the topic comes up it usually pits Democrats against Republicans, with Democrats favoring an overturn of the system and with the Republicans favoring the system in place. While it’s pretty apparent that the electoral college does disadvantage Democrats who are primarily located in more urban states. I was curious to see if it in fact also disenfranchises Republicans as well. The strongest argument that I hear in favor for the electoral college is that it gives a bump to more rural populations who would otherwise never have their concerns met. And it’s typically Republican voices that bring up this concern. Which I don’t fault them for advocating for their consituents. But I’m not sure if addressing that concern on a state level really makes much sense. While our typical idea of a rural voter might be someone living in Kansas or other midwestern states. I think we underestimate how rural/Republican huge swaths of even very populous states can be. So I decided to pull some state level data on the population, number of registered voters, and their partisan composition and see if there is a difference in partisan composition in more populous states compared to the average composition.

Adding user comments to your website

In this post I’ll go over how to expand a basic html serving flask website to a more dynamic website including content that you the web developer has put in and content added by users of the site. While creating comments is the clearest use case for these techniques. The broad strokes apply to most cases when you want information from your front end to communicate with your backend.

Programmatic language aquisition

It dawned on me the other day while talking to someone new to programming that learning a programming language follows steps that mirror human language aquisition.

Transforming Rows into Columns

SQL is pretty straight forward to get into yet despite the ubiquity of SQL Databases, there is comparatively few online resources on how to think about and troubleshoot complex queries. While it’s maybe not the most alluring aspect of data science, it’s pretty darn near foundational. In my current job, basically my entire job in one shape or form goes back to queries to our MySQL Database. While generally you want to design your database in a manner that makes reading and writing to the database easier. You’ll often find yourself in situations where you’ll have to use more sophisticated SQL queries. One common case basically involves transforming data that is held in rows into columns. This can be achieved in a crosstab query.

Introducing Xlwings

In this final part of this tutorial series, we’ll go over how to make a basic yet functional front end for what we’ve created so far in this tutorial series. Typically when we refer to the front-end of our application we are referring to how a user interacts with the program we’ve written. A front end can take the form of a graphical user interface (GUI) which runs on a user’s own machine, or more commonly these days on a mobile platform (iPhone, android) or on a web site as a web application. For data science projects the web app is usually the optimal way to distribute your work out to the wider world. So If you’re looking to bee line to that step I’d suggest looking into Flask for the web development aspect and heroku for deployment.

Data and Visualization with Jupyter

Jupyter notebooks were originally conceived of as a portable means of combining code and written research, and in my opinion an excellent environment for exploring data and toying around with different data engineering operations.

Performing Descriptive Stats with Python

In my previous post we started with downloading and installing the Anaconda python distribution and loaded our health data CVS into a Python interpreter. I know that a lot of that installation work and perhaps figuring out pathing may have been frustrating. But like anything new we do, those initial challenges and frustrations are what build experience.

Getting started with Python

For this series I’m going to approach introducing the tools and techniques of Data Science, in a manner I’ve so far not seen online yet. Most introductions to Data Science revolve around datasets that are already well known and with the foreknowledge of what can and cannot be extracted from it. A certain 20Th century tragedy and various flower specimens come to mind. While there’s not exactly wrong with learning data science through these kinds of examples, I think it removes the element of data science I personally love the most: discovery!

Getting started in the Programming World

There are a myriad of ways to get started programming and in general I don’t really prescribe to any notion that there’s one or even a few right ways to start out. What I think is important is that you start getting into a process of producing code and seeing the results. And I think the best way to do that is to learn to code via projects. I’ve heard from countless people that they tried learning to program but quickly dropped off because they spent hours and hours writing scripts that really didn’t do anything. I don’t think it’s their fault but a lot of programming tutorials out there seem to be fixated on covering all the basics until they moving on to producing actionable artifacts. I’m sure for the more orderly left brain people that makes sense, but for me personally no amount of will power is sufficient to keep me doing something that doesn’t have a clear goal to strive for. What follows are a few options for what I think are pretty reasonable paths that I know of to get you programming and more importantly creating something that either you or others will find useful. I am primarily interested and practice Data Science but like many newer programmers I do a fair amount of web development work. So what I’m going offer are my recommendations for what to cover when starting out in these two fields.

It’s really just about typing

Everyone should program, or well perhaps not everyone, just as not everyone should ride a bike, or drive a car, or browse the internet. Nor will everyone program even if they could, just as not everyone wants to read a book or figure out how to pay their taxes online. But in this day and age, we take it as given that most functioning adults should gain skills such as the ability to drive a car, be literate, or put together a PowerPoint presentation. In a historical sense all of the above used to be exclusive domain of specialists, who were privileged to know such elusive arts such as the how to read the written word, operate a model-T or type on a typewriter. But as the utility of such skills grew in an economic sense, skills such as typing became an assumed skill. And people, as they always have done adapted. And as more people adapted the adaptation process for others became easier. As reading and writing became more important, society produced teachers to educate the masses. When cheap automobile production was possible, not only did cars proliferate but more user-friendly features for using a car proliferated. Automatic transmission wasn’t a thing until a gear system was invented.