Been working on a little something for comparing JSON data between two files that have the same structure. I use this mainly for when I need to check for inventory updates from a database on a monthly basis and don’t want to manually load the data into something like Excel, and instead stay in the commandline. I prefer the latter because it’s easier for me to document and archive work, which is much easier when months later a review is needed to audit the data quality.
Anyway, I’m writing two parts. One to go over how to read JSON files with Aeson, and another to go over comparing JSON files.
Writing in Haskell as usual was an exercise of persistence, a whole lot of research, and comprehending the available documentation. Working on a specific use case is a great way to learn a little bit more about working with Haskell.
I chose the Aeson library because it’s what I worked with before when reading JSON APIs and I thought it’d be a good place to start. There’s probably more libraries available, I haven’t yet had a chance to review all of them.
Part 1 - How to set up Aeson
I’ll briefly cover what Aeson is and how you can set it up to read JSON text files.
Dependencies
The script uses the Aeson Library, which you can import after installing it like so:
import Data.Aeson
import qualified Data.ByteString.Lazy as B
Library imports go towards the top of the file, you can import them with a variable name. In addition to Aeson, I’m also importing ByteString because I’ll be reading from text files.
As for how to install, there have been many tutorials written, I’m not too interesting in re-inventing the wheel so will link you to Stack.
Note: At the time of this writing, Aeson is currently at version 1.3.0.0.
Reading from text files
Reading in a text file is as simple as using the readFile function to read in the whole contents. Then with this data I used a utility function to parse it.
a <- B.readFile "foo.json"
The json file contains a list of dictionaries (see below) where all the dictionaries have the same keys. This consistency is important for this exercise as we move along to abstracting the structure.
[
{ "foo": 1, "bar": "baz"},
{ "foo": 3, "bar": "foo-baz"},
{ "foo": 4, "bar": "foo foo"}
]
turns into
"[\n { \"foo\": 1, \"bar\": \"baz\"},\n { \"foo\": 3, \"bar\": \"foo-baz\"},\n { \"foo\": 4, \"bar\": \"foo foo\"}\n]\n"
readFile not the only way to read a text file, but this works for me in providing the json files in a format that Aeson’s eitherDecode could use.
Parsing
eitherDecode a :: Either String [Value]
eitherDecode docs has the type signature eitherDecode :: FromJSON a => ByteString -> Either String a which takes the ByteString and returns an Either` type. I like to think of this process as using bubble wrap to safely handle the delicate contents with a fragile sticker.
Then to separate the error message from the contents is a matter of separating right and left. The decoder function below I wrote to take out the right side.
decoder :: Either a b -> b
decoder (Left _) = error "Error"
decoder (Right x) = x
let r = decoder (eitherDecode a :: Either String [Value])
The JSON file was a list of dictionaries, so now we end with a list of record types. Haskell doesn’t have the exact same concept of dictionary so this record type is used instead.
What I love about using Aeson here is that it can intuit the JSON structure - I don’t need the level of specificity saying what each type is within the list but it has been identified anyway.
[Object (fromList [("foo",Number 1.0),("bar",String "baz")]),Object (fromList [("foo",Number 3.0),("bar",String "foo-baz")]),Object (fromList [("foo",Number 4.0),("bar",String "foo foo")])]
See how it looks in action!
Printing results
Show the output with show
Prelude.map (\x -> show x) r
Next in part 2, I’ll show you how to compare two json files!