Categories
Patents

Patent: Discrepancy Resolution Processor and Methods for Implementing the Same

This is a patent I was issued for a system I wrote while on the prototyping team at Salesforce.

While we were creating prototypes, Sketch would often export SVGs that (once loaded in the browser) had invalid html with weird artifacts and didn’t quite match up with what the designer was seeing in Sketch. The PNG files that Sketch exported would match. So what I did was created a Sketch plugin that would export both an SVG and a PNG, and then load the SVG inside of a browser (Puppeteer), clean up the HTML, take a screenshot, and compare the screenshot to the PNG (using ImageMagick). Depending on what it saw, it would make further modifications to the code within the SVG file (then Puppeteer, screenshot) and compare the modified SVG file to the PNG file, and notify the designer about the outcome.

Here’s the relevant text from the patent:

Discrepancy Resolution Processor (DRP), System and Method for Programmatically Comparing, Fixing, and Flagging Discrepancies Between a First Exported Image File Having a Raster-Based Image Format and a Second Exported Image File Having a Vector-Based Image Format

A system is provided that can execute one or more scripts to cause a discrepancy resolution processor to be executed on a first exported image file (having a raster-based image format such as PNG), and a second exported image file (having a vector-based image format such as SVG) to programmatically compare, fix, and flag discrepancies or differences between the first exported image file and the second exported image file.

In some embodiments, a DRP and associated methods are provided. The DRP can, for example, programmatically compare, fix, and/or flag discrepancies between two different image formats of exported visual representations of the same subject matter, such that each is appropriate for display in a web browser by comparing them for a “sufficient” threshold of visual similarity (e.g., that is configurable and acceptable similarity). For instance, in one non-limiting implementation, the DRP can, for example, programmatically compare, fix, and/or flag discrepancies between a raster image format (e.g., PNG) versus a vector image format (e.g., SVG) of exported visual representations of the same subject matter. In this example, discrepancies between the two different image formats are resolved so that each is appropriate for display in a web browser with acceptable similarity (e.g., are within a defined threshold of visual similarity that is acceptable or “sufficient.”).

In one embodiment, a server system is provided that includes at least one first hardware-based processor and memory. The memory comprises processor-executable instructions encoded on a non-transient processor-readable media. The processor-executable instructions comprise one or more scripts. When executed by the first hardware-based processor, the processor-executable instructions are configurable to cause the discrepancy resolution processor to be executed on a first exported image file (having a raster-based image format such as PNG), and a second exported image file (having a vector-based image format such as SVG) to programmatically compare, fix, and flag discrepancies (e.g., differences) between the first exported image file and the second exported image file. The second exported image file has a format that is subsequently programmatically manipulatable including one or more of: interpreting the image data as a Document Object Model (DOM); and manipulating the image data via JavaScript or CSS.

In accordance with this method, the first exported image file can be resized so that dimensions of the resized version of the first exported image file are the same size dimensions as those of the second exported image file.

The second exported image file can be cleaned up to make the second exported image file appear visually similar enough to the first exported image file so as to fulfill the needs of the software demonstration being created. For example, cleaning up the second exported image file, can include one or more of: renaming identifiers of the second exported image file to ensure that each is unique and do not collide with other identifiers of other SVGs that on a webpage; cleaning identifiers of the second exported image file so they are valid for Hypertext Markup Language (HTML); and performing other manipulations with respect to the second exported image file to optimize use of the second exported image file in a web browser. The other manipulations of the second exported image file can include one or more of altering objects of the second exported image file so that they are positioned, sized, oriented, colored, and/or shaded, in a way that more closely matches their positioning, sizing, orientation, colorizing and shading in the first exported image file.

After cleaning up the second exported image file, the second exported image file can be loaded as a Document Object Model (DOM) so that a script can interact with the DOM as if the second exported image file were in a browser.

Problems (or problematic characteristics) with the second exported image file that resulted when the second exported image file was exported from image editing software can be identified, and any problematic characteristics of the second exported image file that are identified can be adjusted. Examples of problematic characteristics associated with the second exported image file can include one or more of: text sizing being incorrect; text spacing being incorrect; leading; word spacing; letter spacing; border width; and drop shadows missing; and component positioning on the page being placed incorrectly. For example, if an element has a line-spacing specified on it, the script mathematically calculates where the element ought to be (based on font-size and line-spacing) and places it there instead of where the image editing software put it. As another example, the script can adjust an element’s font-size and letter-spacing based on a formula, and/or adjust an element’s font-weight based on a formula, etc.

The second exported image file can be inserted within a Hypertext Markup Language (HTML) shell that mimics a final context that the second exported image file will be loaded into. The second exported image file can be rendered in the browser and a screenshot of the rendered second exported image file can be taken.

The resized version of the first exported image file and the screenshot of the second exported image file can be processed to determine the discrepancies between the resized version of the first exported image file and the screenshot of the second exported image file. For example, the resized version of the first exported image file and the screenshot of the second exported image file can be processed via a diff program that determines the discrepancies between the resized version of the first exported image file and the screenshot of the second exported image file to generate a difference file that records the discrepancies.

By comparing the discrepancies to a configurable threshold, it can be determined whether an image of the resized version of the first exported image file and an image of the screenshot of the second exported image file are similar enough, according to the configurable threshold, to be considered to match.

When the image of the resized version of the first exported image file and the image of the screenshot of the second exported image file are similar enough to be considered to match, the matching images are placed into a success folder (e.g., placing difference files of the matching images into the success folder). When the image of the resized version of the first exported image file and the image of the screenshot of the second exported image file are not similar enough to be considered to match, the non-matching images can be placed in a failure folder.

In some embodiments, the discrepancies between the resized version of the first exported image file and the screenshot of the second exported image file can be highlighted in any way that is visually detectable so that the discrepancies are clearly visible to a user. In addition, a visual representation of discrepancies that can be created that cycles back and forth between the resized version of the first exported image file and the screenshot of the second exported image file so that the discrepancies between the resized version of the first exported image file and the screenshot of the second exported image file are observable when viewed by a user. The visual representation could be any way to quickly communicate discrepancies or differences between the resized version of the first exported image file and the screenshot of the second exported image file so that the discrepancies are observable to a user (e.g., an animated GIF file).

One possible non-limiting embodiment will now be described with reference to FIGS. 13-15. In this embodiment, a system and method are provided for programmatically comparing, fixing, and flagging discrepancies between PNG and SVG export files; however, it should be appreciated that the discrepancy resolution processor and similar methodologies can be applied to other types of image files (e.g., exported image file having a raster-based image format and another exported image file having a vector-based image format) to programmatically compare, fix, and flag discrepancies (e.g., differences) between the exported image files. An SVG file is a graphics file that uses a two-dimensional vector graphic format created by the World Wide Web Consortium (W3C). It describes images using a text format that is based on XML. SVG files are developed as a standard format for displaying vector graphics on the web. As is also known in the art, a PNG is a raster-graphics file-format that supports lossless data compression. PNG was developed as an improvement for Graphics Interchange Format (GIF). PNG supports palette-based images, grayscale images, and full-color non-palette-based RGB or RGBA images.

Leave a Reply

Your email address will not be published.