In an effort to make the MetaSharp pipelines more powerful I’m about to add the concepts of stages and connectors. I’ve been thinking about it a bit and I drew up some diagrams to help me express how the pattern should work.

At a high level it’s pretty simple, for every pipeline there are multiple stages and for each stage there are multiple steps. Each stage has 1 or many input connectors and 1 or many output connectors, which connects to the next stage of the pipeline.

With this in mind there are four possible types of stages, defined by their input and output connectors. Stages must be chained together with matching input and output connections. You want multiple types because there are certain types of operations that are simply not possible to do simultaneously but there are other types that are completely isolated and are perfectly acceptable to run asynchronously.

Many to Many

For each type of input a complete inner pipeline of steps is created. Meaning each input value from a previous stage will be processed by the same steps. Each inner pipeline will run asynchronously and should not communicate between each other. The stage will complete when all steps have completed running.

1 to 1

This type of stage will accept one input value and produce one output value. It will create exactly one chain of steps and execute synchronously.

1 to Many

This type of stage will accept one input value and have exactly one chain of steps but will produce many output values.

Many to One

This type of stage will accept many values and run them all through exactly one chain of steps.

From this I should be able to make any type of compilation pipeline imaginable. For example a typical pipeline might be something like this:

• Parse files
• Combine AST
• Resolve References
• Generate Assembly

In which case you might end up with the following stages:

• M:M, Parse files all at once
• M:1, Combine the ASTs into one tree.
• 1:1, Resolve and transform the tree.
• 1:1, Transform into IL

You could also imagine that last step transforming into multiple objects or multiple files or something like that quite easily. Also the good news is that I think this shouldn’t actually be that complicated. The pipeline simply deals with connecting stages and each stage has a very simple strategy for processing the steps. The real work will lie in the implementing the stages but even then each stage is completely modular and singularly focused.

I’ve been thinking about how you might layer DSLs into ever increasing abstractions and how you might combine multiple DSLs with ease. These are problems I definitely don’t have the answers to just yet but while looking at how Axum is coming along I can’t help but feel like Axum is a language designed specifically to help you write code that conforms to a specific design pattern. I mean it’s still .net under the hood (I think) which is no more or less async safe than what you or I can write manually but it constrains you such that you cannot create bad asynchronous code (or should make it much harder at least!).

This is really fascinating to me, since intuitively it feels like a really good idea but I don’t quite get how it can coexist with other types of development or DSLs right off the bat. I also think it reinforces the idea that constraint, at times, can be more powerful than flexibility. One of the things about “dynamic” languages that I’m not totally convinced about is the how it feels like the wild-wild-west, anything goes type of programming. Sometimes having limiting constructs can actually more powerful and that is not appreciated enough, I don’t think.

Anyway, I was just trying to picture a world where you had one DSL where you designed various models for your application and another DSL where consumed them in a specific design pattern. This seems feasible I’m still trying to figure out how you might “glue” them together. Probably using templates somehow. Food for thought, if anyone has any insight or ideas leave a comment!

There are lots of cool videos of demos and discussions over at Mix09 today. If you haven’t been there you should definitely head on over.

http://live.visitmix.com/

There are lots of cool announcements for new features for the next version of various Microsoft products. Too many for me to even enumerate in a single blog post actually but what’s really exciting is that I finally get to tell everyone what I have been working on these last couple on the job at Microsoft.

I have been working with a small team of developers, here in Minneapolis, who are responsible for the Photoshop Import feature of expression studio. It’s been a very interesting an exciting project so far and it seems like something that developers and designers are both very happy about. Here is a screen shot from the Mix demo site.

We’re able to read the layer information from a Photoshop file and import them as separate controls into Blend. Once there a developer can make those generated layers into real interactive controls like anything else in Blend. There is also a re-import feature to ease to workflow from designer to developer. This exact incarnation of the importer is available in the build of Blend 3 Preview, so download that and give it a shot! Feel free to send me feedback if you do try it out.