Posts tagged: tech

fx is an interactaive JSON viewer for the terminal.

It’s a simple tool built with Charmcli’s Bubble Tea.

Installation #

The installation with go was broken for me - both via the link and direct from the repo. Now I’m not a gopher so I don’t really know how to fix that.

Luckily npm install fx also works and got me what I needed!

Usage #

Usage is simple… fx <json file>. The Github has a few other ways such as curl ... | fx etc.

Type hinting has helped me write code almost as much, if not more, than unit testing.

One thing I love is that with complete type hinting you get a lot more out of your LSP. Typing dictionaries can be tricky and I recently learned about TypedDict to do exactly what I needed!

The Problem #

It might not be straight up obvious what the problem is, especially if you don’t utilize tools like mypy or flake8 in your development.

My handy-dandy nvim-lsp gives me a lot of feedback when I’m coding and it’s immensely helpful.

So with the LSP giving me constant feedback here’s the issue:

from typing import Dict, List, Union

my_dict: Dict[str, Union[List[str], str]] = {
    "key_1": "val_1",
    "key_2": ["ls_1", "ls_2"],
}

my_dict["key_2"].pop()

With the above script you’ll get an annoying warning about using pop on key_2.

The Solution #

Maybe you can stomach getting yelled at by your LSP but I like complete silence if at all possible.

TypedDict was the saving grace.

from typing import TypedDict

MyDict = TypedDict("MyDict", {"key_1": str, "key_2": List[str]})

my_typed_dict: MyDict = {
    "key_1": "val_1",
    "key_2": ["ls_1", "ls_2"],
}


my_typed_dict["key_2"].pop()

I was able to import TypedDict from typing, mypy_extensions, and typing_extensions

With TypedDict you define your custom type, match the first argument to TypedDict with the name of the variable (idk why), then type hint each key you expect in the dict! It’s super easy and I think puts you into a position of being extremely explicit with your dictionary variables. This isn’t always desired or appropriate but in most of my use cases it is.

RTFM #

There’s other implementation of TypedDict and while writing this I saw that most of the docs define a class for the type like this:

from typing import TypedDict
class MyDict(TypedDict):
    key_1: str
    key_2: List[str]

my_dict : MyDict = {'key_1': 'val_1', 'key_2': ["ls_1", "ls_2"]}

pep docs

mypy docs

I have a specific need for counting the number of lines in a file quickly. At work we use S3 for data storage during our Kedro pipeline development, and in the development process we may end up orphaning several datasets. In order to keep our workspace clean I have a short utility that compares the datasets in a Kedro DataCatalog with the files in the relevant S3 location.

To get that list I run an internal tool like this:

kedro our-liter | grep s3 >> orphaned_datasets.txt

This simply parses our internal linter for the lines releated to my s3 linter utility and pipes those lines to a file.

To get a quick idea of how out of wack a pipeline is I could open the text file in vim, git it with the G and see what line number I’m on but I’m way too lazy for that…

AWK #

awk 'END {print NR}' orphaned_datasets.txt gives me the number of lines and I can alias this to whatever feels appropriate in my zshrc!

built-ins for the win!

Did you know you can spell check in Vim?!

Without... #

Here is a missspelled word.

  <h3>With!</h3>
  <p>Here is a <u>missspelled</u> word.</p>

What is this magic??? #

set: spell spelllang=en_us

Custom words? #

Sometimes there’s things that are words to you but not the default spell checker…

Common example: package names!

plotly, streamlit, psutil, etc etc…

You can easily add these to your vim config by hitting zw ontop of the word!

I am working on a project to create a small system monitoring dashboard using the python psutil library.

The repo is here (if you want actual system monitoring please use netdata).

I’m using streamlit and plotly for the webserver, design, and plotting at the moment.

My Use Case #

I needed a way to refresh my plotly charts with a fixed window of time so that I’m able to just see relevant recent data instead of cramming all data for all time into one plot that’s 500 pixels wide…

Checking the length of arrays or lists every time I get a new piece of data feels kind of dumb and I thought “python must have a way to do this”…

“This” meaning, update values in a fixed length array without reallocating memory or recreating a copy of the list

Deques #

Enter the deque. It means “double ended queue” and is in general an Iterable that you can append values to either side or pop values from either side.

The init signature is straightforward enough and I’m sure there’s more to them than I know yet but here’s how I use it…

from collections import deque

my_deque = deque([1,2,3])

This gives us my_deque, created from an iterable, with several familiar methods like index, extend, append, etc. However there’s some new ones too such as appendleft and popleft.

my_deque.appendleft('a')
print(my_dequqe)
>>> deque(['a', 1, 2, 3])

my_deque.popleft()
>>> 'a'

<!--markata-attribution-->
print(my_deque)
>>> deque([1, 2, 3])

These are handy ways to manipulate the iterable that I needed for the arrays I plot with plotly!

See my follow-up to this on using Deques with plotly and streamlit to create a quick “dashboard” with live streaming data!

follow-up