Removing the text/dataset system

#1

@ryan and I are strongly considering scrapping the dataset-based text system, and we'd like ya'lls thoughts.
tl;dr: We want to focus on layout and playfulness, and cutting the complex text system will help us do that.
We'll revisit data/text later, once we have the resources to do a great job on it.

Here are our reasons for scrapping the current text/dataset system in the near term:

  • The next release will be a breaking change because of the new layout engine, so now's our chance to get rid of the dataset stuff.

  • The text dataset stuff is half-baked anyway --- to really fulfill the dream, it needs to pull in text from outside sources (APIs, spreadsheets, etc.), and those changes would likely be breaking anyway.

  • Removing the dataset stuff will likely make Subform faster on text-heavy designs.

  • The current UI doesn't do a good job of making the capabilities and semantics clear, and would require substantial work to do so.

For example, when creating a new component instance, all text fields are filled with the first datum of their respective dataset. This is because, unlike Sketch symbols, there is no "canonical" instance of a component that specifies which datum should be the "default" for a given text item. Most people find this behavior surprising (topic). These semantics actually caught @ryan by surprise when he was making this video for the website/Kickstarter:

Creating a new "drop down" component had every result item set to "New York", which he didn't understand. And if you don't understand the thing you helped design/create, it's not a good sign = )

  • Ripping out the dataset stuff will take far less time than cleaning up the existing system and writing documentation/tutorials/UI changes to teach its capabilities.

  • The dataset stuff was least popular feature in the survey ya'll took.

We'd replace the text system with something very similar to Sketch-style overrides: You create text boxes wherever you like, and if they are within a component you can override the text on specific instances.

We'll need to make a decision in the next two weeks so we can integrate the text with the new layout engine before cutting the next release.

To be clear, we don't want to scrap the datasets because they're a bad idea --- we still strongly believe that designers need to work with real data during their design process, think about the "role" text plays in designs rather than the text itself, and that this process/thinking can be helped by better tools.
But right now our implementation sucks and trying to improve it will detract from more important capabilities (layout, playfulness, performance, etc.).
So we'd like to scrap datasets in the near-term and revisit the question later once we have the resources to do a great job.

What do you think?

2017.01.17 Status update
#2

The Sketch style text override is sufficient for the design cases our design team has to show in our e-commerce products.

The Craft plugin gives the optional advantage to fill a list of strings into user managed collections, which can be applied to text boxes with one click. We used this but dropped it after a couple of weeks, since once you built your components with real data there was no need to use the data plugin anymore. Also, the generic pre-populated datasets seem great and useful at first, but I almost always come to a point where I need more specialized strings. At that point the templates become useless and creating a special dataset is too much of a hassle.

I think the theory of a feature-rich data populating system is really compelling, however in (our) production reality speed and ease of use always tops deep content management. If today I come across a new string of text to test I just drop it into the sketch override or into the prototype HTML.

What I would suggest is to experiment using variables in overrides. If I can edit an array in a single text field, I only have to set up a repeating element once and skip a massive amount of clicking and selecting in my canvas.

tl;dr: drop datasets, allow overrides, focus on layout :slight_smile:

#3

Among the many things I like about Antetype are that text is simply a property of an object. Thus, it can be overridden per symbol/widget instance just as any other property can. And, it can be set to be part of the "canonical" definition of a symbol/widget, just as any other property can.

Also great is that you can make these overrides INLINE, not in a confusing list of textboxes.

#4

@kevin, I agree with scrapping the current "dataset-based text system" and replacing it with the Sketch-like override system for now in order to improve the other important features you mentioned as quickly as possible, as long as we do revisit a better way of using real data in the new layout system before the full release of Subform.

#5

@David, can you describe your use case in more detail?
Not the imagined implementation, but what specific sorts of problems you are trying to solve:

  • populating a table with stub data?
  • interpolating variables within a string (e.g., turning "Hello {{NAME}}!" into "Hello David!")?
  • keeping language across multiple screens consistent?

I've thought about allowing variables in text (and perhaps other fields, like color or font family), but it touches on a ton of different issues.

Take this example from Angular.js:

var exp = $interpolate('Hello {{name | uppercase}}!');
expect(exp({name:'Angular'})).toEqual('Hello ANGULAR!');

What is happening here is that the string "Hello {{name | uppercase}}!" is interpolated to become "Hello ANGULAR!" because there's anamevariable in scope that is passed to a functionuppercase` that returns "ANGULAR".

Already we have syntax for interpolation, {{ and }}, function invocation ,|, and some unexplained notion of scope (i.e., where name and uppercase come from).

Your example of wanting to iterate over an array is a common need, and trying to handle it within strings is complicated.
Here's another example from Angular:

<div ng-repeat="n in [42, 42, 43, 43] track by $index">
  {{n}}
</div>

It looks like n is a name that takes on the value of the current item (presumably shadowing any existing n in the broader scope) in the array (which magically gets evaluated, assuming that [42, 42, 43, 43] is a literal array value rather than a name in a higher scope), and by $index is probably some kind of key that provides an identity to the interpolated {{n}} children.

I wouldn't be surprised if this expression can get even more complicated by adding filters or something to the array.

Anyway, my point is that variable interpolation systems can quickly become, If you're not careful, a ghetto, string-embedded programming language full of UX confusion, security holes, etc.

We need to understand the most common use cases / needs so that we can come up with solutions without falling into the unfriendly "power user" nerd trap. = )

#6

I like the reasoning behind the current text system, but I agree that the current implementation is clunky at best, and frustrating/confusing at worst. I'd support taking it out for the time being. As long as you can do instance overrides that should be good enough for now.

#7

If I understand correctly you worry what might happen if a user creates an unexpected result by mixing multiple variables and arrays (sorry, I don't know angular code).

Avoiding conflicts should be solved by isolating user generated variables and arrays from the rest of the code. Webflow for example sets a prefix (w_) before every class the user creates. Also I would not recommend allowing variables in arrays at all, to keep the system simple. Text replacement should not become a deeply complex thing.

To answer your question:
We use text overrides in all our repeating elements, to test varying string lengths. Take a card for example:

It has a title (Top 10 Australian beaches), a subline (Number 10), copy text (Whitehaven ...) and button text.
Usually us designers have a wireframe and content guideline beforehand, so we know that these types of text have to appear in the card. These of course can have wildly different lengths, from 2 words to 2 lines each, except the buttons. The UI has to adjust to the content, not the other way around.

The system that makes our life easier would allow us to create the element and assign variables to the kinds of content, like $card-title, $card-subline and $card-copy. Calling array values could be done by writing $card-title(random) or $card-title(6) in the override field.

Here is an ultra quick wireframe of how I imagine the management UI:

Pasted image312x592 16 KB

EDIT: oops, forgot a field for variable value. In fact, you would only need arrays, because variables are just arrays with one entry.

All just my quick, untested ideas and wishes. I'm sure Ryan will come up with a more elegant UI for that :wink:

#8

Thanks for drawing up this example, @David.

Kevin and I do a lot of paper prototyping when we're thinking through semantics and it usually looks a lot like this. Having a concrete example and rough UI makes it a lot easier to think through the application behavior.

Do you have any thoughts about how you'd want to organize your data when datasets get large? It's easy to think about one map of "titles" and another map of "subtitles", "copy text", etc. Does everything get thrown into a few big maps—or should some titles only apply to certain UI elements/components? In other words, should there be "title_beaches", "title_cities", "title_mountains" maps?

When we're talking about working with "real data" things can get complex quickly. Not just in terms of raw amount of data, but also different presentations of the same data: languages, only allowing certain variables to be used in certain places, etc. I think some of Kevin's concerns from above are around namespacing, manipulation, and such—things are are really the domain of programming languages. Do we build visual tools for data manipulation and management into Subform?

#9

I'll try to answer your question about larger datasets with an example.

Since I'll be getting kinda deep into workflow, I'd like to say that it is very likely that my style of working is completely incompatible with someone else's situation at their job. Also I probably made an error along the way since it's close to 1:30 am here.

 

Let's analyze the spotify UI.

blob.jpg2559x1558 653 KB

 

When I build a UI like this for production and maximum adaptability, I cut it down according to Brad Frost's atomic design model. What this means is that you cut down your UI to the smallest element, then group these into bigger and bigger chunks. The chunks can be repeated and placed however you like, just like LEGO.

Almost every UI is just a collection of repeating elements with different states.

Here are the atoms:

blob.jpg1354x780 338 KB

The molecules:

blob.jpg1372x779 314 KB

The organisms:

blob.jpg1383x779 288 KB

Now let's say we learned something new from user research. The social stream on the right needs two rows for the song title instead of one. Here's what I would need to do to show that:

In sketch (or figma)
I would have a symbol/component ready called "social_list_item", which includes the
- avatar Image
- person name
- song title
- icon currently listening
- time last listened to
- icon type (I think?)
- album name

The steps to change the layout are:
- edit the symbol/component to show two lines of text
- pixel-push the album name down
- pixel-push all the other social_list_items down to match the new height (or use the butter plugin for sketch, which takes some of the pain out of this)
- Select first symbol
- copy new album title from my reference
- paste new album title into override field
- repeat for each and every one, everytime we need a change.

If I had a simple multi-line textbox with 10 album titles in it, I could populate each occurrence just by changing the content there. In my symbol I would just put into the override field $song_title(random).

I hope this example shows how I would organize my data. In short, control of multiple content elements scattered across the layout file from a single point.

I had a thought that symbols can also benefit from an array system, but right now can't put it into words. I'm eager to hear other opinions on overrides!

#10

That clarifies your workflow/needs a lot, David. Thanks again for taking the time to write this up in detail.

#11

@kevin you asked about parametrized components (in this thread). I think that one way to implement them would be something similar to Apparatus by Toby Schachman (he’s working in Human Advancement Research Community with Bret Victor and others (I heard that Alan Kay is also involved somehow) on inventing the new ways for human-computer interaction).

You can describe pretty complex relations with it (they even have Quadratic Bézier, bicycle, and inverse kinematics in examples).

And that will work until you decide you need some events and logic :slight_smile: But I’m not sure, what will be the best way to deal with such things. May be you’ll need something like Noodl (which works on several devices, has API and possibilities to extend it with JS, and can use external data) to manage that. Or just use something similar to Axure. It's a big topic anyway.

Hope that was useful.

#12

I've seen Apparatus and I know a few of the people over at HARC/Viewpoint --- they are definitely working on some cool stuff.

As you implied w/ events and logic, it's not obvious at what point to stop.
To say, "yes, I understand --- you can express that idea in English, but you cannot express it in this tool. Use something else".
Ryan and I have kept Subform focused on visual design (as opposed to animation / "prototyping"), and even then it's not clear how much needs to go into it, hence this thread = )

I hadn't seen Noodl before, thanks for the tip on that.

One point I want to make about Apparatus and Noodl ("the most powerful codeless tool to build user interfaces for connected applications") is that they allow for expression of complex logical/mathematical ideas using non-textual interfaces.

There is lots of prior art in this space (MaxMSP, LabView, etc), and visual interfaces work very well in some domains.

However, they're not always better in all respects than textual interfaces, no matter what various Kickstarter projects might wish you to believe.
While there is definitely a conspiracy of neckbeards that keep programming inaccessible via forceful lack of empathy, there are also other reasons why many people prefer the command line over GUI wizards.

Furthermore, the interface (graphical, textual, or otherwise) can only be as powerful as the underlying model permits.

The "you can override text values in component instances" semantic can be exposed both within a graphical interface (swatch on the sidebar) and within a textual interface (hotkeys / smarty bar).
But those semantics have fewer concepts than something like the original text / datasets system that we shipped in the first releases of Subform.

While it's worth thinking about how you make concepts discoverable within interface(s), it's also worth thinking about the concepts themselves: What problems does this concept solve? What problems does it interfere with? Can we design a interface/system where the user can ignore this concept, or is it essential to understand for all tasks? etc.

Those are the kind of questions I try to figure out first, then I worry about the textual syntax / Kinect integration / Hacker News demos = )

#13

+1 with David on this. This is how our team works (or tries to work at least – time and budget dependant) on every project.

I'd be sad to see the dataset go as it's one of the big things that drew me to Subform. With our clients they almost always want to see the exact final text put into the designs before signing off for build so we spend many many hours each week updating designs with everything from typo fixes to whole new articles being swapped out. So having an easy way to quickly change all instances of a certain piece of text (whether that's an API, spreadsheet etc) would be a huge time saver for myself and my team. I know the current version is like you say, not fully-baked so I'm all for it being re-thought out and improved upon but I would hate to see it fall to the back of the queue.

#14

Hey guys. I wasn’t able to read all the posts from start to the end, but briefly checked what you are talking on this thread. The idea of using variables for any component/instances properties is very good implemented in https://www.paintcodeapp.com/ They allow you to create on global document level various variables of different types - strings, numbers, bools, colors (!) - and then reference to these variables from any properties of any design elements where applicable. For example, you can set manually background color of any box, or you can set it equal to a variable, and then later you change your variable value and it will affect all your designs where you used this variable. Same goes for text, numbers, etc. You can even use it in expressions to define calculated values for parameters of your design elements. You can see more how they work in tutorials https://www.paintcodeapp.com/tutorials Having same functionality in Subform (including ability to set fonts via variables) would be fantastic!