The workflow to add new building blocks into the editor’s collection is the following: the design team produces the images of the interfaces they want to implement (see examples in the challenge forum); these images are shared with the dev team that has to break those design into re-usable components and convert them into JSON description in the format the editor can undestand (see an example in challenge forum). The client wants to explore the possibility to automate this process, and if we prove it is feasible to produce a proof-of-concept (PoC) app showcasing the approach in action.
We start with the ideation challenge. We want you to explore and suggest the best algorithms / libraries / software tools, that will help to get the design images as input, to analyse them, recognizing the hierarchy of HTML components present in them (from complex as menus, data cards, etc. to the primitive ones like buttons, switches, input fields), and output the JSON description of those objects and their hierarchy. The exact format of the JSON output is not important, as long as it clearly describes the structure of the page depicted in the image. Ideally, the approach should also capture and document the styling details, but if it does not look feasible, it might be enough to automate the process as much as possible, delegating some operations to the developers using the tool; e.g. our PoC app will break the design into components and builds their hierarchy, then it will just highlight detected elements for the user, helping him to rapidly fill in any style details that are hard to capture automatically, and to correct any mistakes done by the algorithm. As long as your solution is able to speed-up the current design/development process described above, it might be interesting for our client.