stz iterating #2

Now that we've solved the syntactical problems of iteration we can circle back to some of the other ideas before we embraced python comprehensions. The one in particular I'd like to look at is composition of blocks.

We know we can use the parameter type → return type to indicate the intention of the operation. If the first block returns a boolean and the next block takes the parameter of type of the first block we know it is a filter operation. Otherwise it is a map operation.

people · [age ≥ 18] · [address] · [println] · stdout

You can't get much simpler than that. A person is pumped in to a filter test, then in to a map test, then in to a println and those strings are pumped in to stdout. Source to Destination is cleanly mapped out and an easy to understand pathway.

The real question is what kind of syntax is desirable. I love the cleanness of · but it's inconvenient to type. Pipe | is safe to use here because we're in an expression and not the start of a block. Let's see how it looks on our eyes:

people | [age ≥ 18] | [address] | [println] | stdout

It's familiar if you know shells, but it looks a little bad next to all those square brackets. I don't think it fits the idiom well enough for stz.

→ is already used to indicate "input parameters flow to an output" so we could re-use that concept.. let's see how that looks:

people -> [age ≥ 18] -> [address] -> [println] -> stdout

That's okay. It lacks the elegance of the · though. We could treat it as a list. That would require us to send some kind of evaluate message to it though. Let's try it to see how it looks:

(people, [age ≥ 18], [address], [println], stdout) evaluate

No. That doesn't look like a 'flow' of information. It looks like a list. Because it is a list. We could instead go with a 'fun' syntax:

people ~> [age ≥ 18] ~> [address] ~> [println] ~> stdout

It's clear something exciting is going on. It is easier to type than · and serves the purpose of making it clear information flows in a particular direction. Speaking of direction - are we doing it in the right direction?

stdout <~ [println] <~ [address] <~ [age ≥ 18] <~ people

I actually found that very hard to type. I don't know why. It's just a flip of the previous one. This works in that assignment is on the left hand side. But we evaluate from the left so in that respect it doesn't work. I think we'll stick with "going to the right" and for now we'll accept ~> as the syntax for describing a flow.

The next thing we need to consider is how we handle errors. If any of these steps fail - such as a person who has no address - then println wouldn't be able to run and we'd have a problem. Usually in functional languages you get back a Maybe from each of these functions and if that maybe is not true then the chain stops.

But that swallows up an error and we need to know about these things. We could make stdout accept a real result and stderr accept an error result, in which case we'd write this to print out the errors:

people ~> [age ≥ 18] ~> [address] ~> [println] ~> stdout ≁> stderr

(For want of a better symbol I used ≁ but something would be used to indicate we want to be called for the failure result not the success result)

But one of the driving design decisions has been to use return values to indicate status with 'output' parameters. This entire chaining design neglects that. It uses an implicit return and an implicit parameter and if we were to add an in/out → status we'd no longer be able to do the short hand.

This strongly pushes us toward using a maybe type solution. It potentially informs us about a lot of other APIs we would design to use maybe as the preferred solution. After all there's very little difference between these two pieces of code:

status := input next-into: &output
status == error then: [abort <- status]

output? := input next
output? else: [:error | abort <- error]

Except that the latter can be even simpler to write:

output := input next else: [:error | abort <- error]

Let's circle back around to the question of 'how do I iterate a collection of people and do something to them?' Well bizarrely instead of having any kind of for loop or do: message we now write:

people ~> [person | marshal mark-inspected: person]

Or if we wanted to copy one array to another array:

people ~> copy-of-people

Or if we wanted to println the first five people (or as many as we have):

people[::5] ~> stdout

(I went ahead and decided if stdout receives an object instead of a string it'll send println to it, because that is a more than reasonable thing for it to try and do)

If we wanted/needed the key and value or index and value then we can write a fuller block:

people
  ~> [index, person |
        ([1000:2000] includes: index)
          then: [person]
          else: [out-of-bounds]]
  ~> stdout

Or we want to write all the people from a database to a json file:

people := database-session read-all: 'SELECT * FROM people'
... people close

output := 'output.json' as-filename open-write
... output close

people ~> [as-json] ~> output

I'm quite satisfied with the result of this exploration for now. It seems powerful an easy to use. Sitting on the notion of using maybe everywhere is important because it will impact the design of the standard libraries completely.

There's a reason that pattern is very popular in a lot of languages. It's got next to no overhead on runtime and guarantees proper handling of errors without any fancy exception handling.