Birth of a power tool: QualifiedItem and the Powershell pipeline

When I was pondering what the oft-requested Powershell interface should look like, one of the main goals was to offer the power of that raw version control API while being easier to use than tf.exe.  Both existing approaches were too clumsy for many of the tasks customers commonly request.  The canonical example seems to be “how do I add the items modified in changeset to a label.”  I once answered a forum post with a Powershell function targeting the API.  Fairly clean, but not exactly concise.  My coworker Mohamed answered a different customer with a tf-based solution.  Good display of his ingenuity, but when I’m using development tools I just want to be efficient, not clever.

for /f "usebackq tokens=2 delims=$" %i in (`tf changeset 1256 /i ^| find "$/"`) do tf label goodbuild $%i;C1256

Yuck (not to mention slow).

And so an idea was born.  Much progress came “free” from Powershell itself: it’s far, far easier to script than cmd.exe is, and when scripting fails, you can drop right into the TFS .Net API without missing a beat.  Some more came from conventions -- standardized verbs, standardized parameters, etc. – and examining how tf commands should be merged, split, or otherwise molded to fit them.  (I admit to being dragged kicking & screaming through this process!)  But ultimately, the biggest value would come from the ability to seamlessly string together multiple commands.

How to reconcile these goals?  To retain their power, cmdlets must output native API objects wherever possible.  The user receives the full data set in structured, strongly-typed form, including the ability to call properties and methods.  Yet for simplicity, the input parameters are rarely more than a couple strings.  In the pipeline I conceived, a form of weakly-typed items are the “glue” that allow rich outputs to feed limited inputs.

Recall that a QualifiedItem represents a simple tuple of {path, deletionID, versionspec}.  By qualifying individual items on the command line, tf.exe commands let you attach a deletion ID and/or versionspec directly to each.  This is the most general way to control the underlying API that tf offers.

The prototype I wrote in early 2007 was just a crude layer on top of tf.exe plus a “magic” extract-items.ps1 script that could turn any suitable object in the public API back into tf’s internal format – QualifiedItem[].  It worked like this:

tfps changeset 1256 | extract-items | tfps label goodbuild

The syntax was nowhere near idiomatic Powershell; the capabilities unlocked by inserting magic scripts in various places were not at all discoverable.  But I was hooked.  Henceforth, I used this little shell as my daily work environment for testing & debugging TFS.  (Direct programmatic access to the data returned by any tf.exe command sure sped things up.)  All the while I collected real-world use cases, tightened/tweaked/fixed, and shopped the demo around in search of a sympathetic ear.  As anyone who’s worked in a huge organization knows, nothing moves without a lot of pushing…and suffice to say that wasn’t my strong suit.  Eventually I did find myself in Brian’s office in spring 2008.  He gave me a green light, conditioned on fixing the fit & finish to Jeffrey’s satisfaction.

Redesigning the cmdlets based on PS conventions wasn’t a technical challenge per se (modulo the kicking :)).  But having to stick a magic cmdlet in between each pipe wasn’t going to fly, no matter its pretty new name Select-TfsItem and incorporation into the main (read: discoverable) C# snap-in.  The final innovation came from Lee Holmes, who suggested we use Powershell’s built-in type coercion to help the pipeline link up automatically: all you need is a constructor on the input type with a single parameter of the output type.  Hyung implemented it as QualifiedItemSpec:

qualifieditemspec

A QualifiedItemSpec represents the collection of QIs extracted from a single input object.  Cmdlets then take an array of these collections as input.  Where appropriate, some cmdlets process each collection as a single entity and yield control; most, however, batch up the entire list of lists and execute a single server call in EndProcessing().  There are also subtleties in the way different object types are coerced into the much weaker QI type.  In the future we’ll look at the various ways such “chunks” of data stream along a TFS pipeline.

Select-TfsItem remains in the toolkit for completeness, but you can see in Reflector that its implementation has degenerated to nothing; the QualifiedItemSpec[] parameter does all the heavy lifting.  It’s still useful for quickly dumping the contents of a huge object like PendingSet into the console or a file.  Meanwhile, let’s see what the final version of our “label a changeset” script looks like:

Get-TfsChangeset 1256 | New-TfsLabel goodbuild

Easy as pie!  Except for the fact that *-TfsLabel commands haven’t actually shipped :)  I coded cmdlets replacing nearly all of tf.exe before leaving, and I know that power tools development remain in good hands, so I’ll leave you with the above example in the hopes it’ll work after the next release or two.

2 Responses to “Birth of a power tool: QualifiedItem and the Powershell pipeline”

  1. BUGBUG: poor title » Blog Archive » Fixing consistency issues with Powershell’s filename properties Says:

    [...] with the TFS Power Tools, Hyung had the foresight to add string[] to the list of types we can convert to QualifiedItems behind the scenes using extra constructors, but we still don’t support the ValueFromPipelineByPropertyName idiom [...]

  2. BUGBUG: poor title » Blog Archive » Get-TfsItemProperty is the bridge between server and local objects Says:

    [...] – comes packaged in the ExtendedItem object.  Even more fortuitously, if you revisit the exploration of QualifiedItemSpec with an eye for detail, you’ll notice that one overload of ToQualifiedItem() is not like the [...]

Leave a Reply