tom’s publications

 

T. Kelly, P. Guerrero, A. Steed, P. Wonka, and N. Mitra, FrankenGAN: guided detail synthesis for building mass models using style-synchonized GANs, ACM Transactions on Graphics, vol. 37, iss. 6, 2018.

Abstract | Bibtex | Project | DOI | PDF

Coarse building mass models are now routinely generated at scales ranging from individual buildings to whole cities. Such models can be abstracted from raw measurements, generated procedurally, or created manually. However, these models typically lack any meaningful geometric or texture details, making them unsuitable for direct display. We introduce the problem of automatically and realistically decorating such models by adding semantically consistent geometric details and textures. Building on the recent success of generative adversarial networks (GANs), we propose FrankenGAN, a cascade of GANs that creates plausible details across multiple scales over large neighborhoods. The various GANs are synchronized to produce consistent style distributions over buildings and neighborhoods. We provide the user with direct control over the variability of the output. We allow him/her to interactively specify the style via images and manipulate style-adapted sliders to control style variability. We test our system on several large-scale examples. The generated outputs are qualitatively evaluated via a set of perceptual studies and are found to be realistic, semantically plausible, and consistent in style.

@article{wrro138256,
volume = {37},
number = {6},
month = {December},
author = {T Kelly and P Guerrero and A Steed and P Wonka and NJ Mitra},
note = {{\copyright} 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. This is an author produced version of a paper published in ACM Transactions on Graphics. Uploaded in accordance with the publisher's self-archiving policy.},
title = {FrankenGAN: guided detail synthesis for building mass models using style-synchonized GANs},
publisher = {Association for Computing Machinery},
doi = {10.1145/3272127.3275065},
year = {2018},
journal = {ACM Transactions on Graphics},
url = {http://eprints.whiterose.ac.uk/138256/},
abstract = {Coarse building mass models are now routinely generated at scales ranging from individual buildings to whole cities. Such models can be abstracted from raw measurements, generated procedurally, or created manually. However, these models typically lack any meaningful geometric or texture details, making them unsuitable for direct display. We introduce the problem of automatically and realistically decorating such models by adding semantically consistent geometric details and textures. Building on the recent success of generative adversarial networks (GANs), we propose FrankenGAN, a cascade of GANs that creates plausible details across multiple scales over large neighborhoods. The various GANs are synchronized to produce consistent style distributions over buildings and neighborhoods. We provide the user with direct control over the variability of the output. We allow him/her to interactively specify the style via images and manipulate style-adapted sliders to control style variability. We test our system on several large-scale examples. The generated outputs are qualitatively evaluated via a set of perceptual studies and are found to be realistic, semantically plausible, and consistent in style.}
}

T. Kelly, J. Femiani, P. Wonka, and N. Mitra, BigSUR: large-scale structured urban reconstruction, ACM Transactions on Graphics, vol. 36, iss. 6, 2017.

Abstract | Bibtex | Project | DOI | PDF

The creation of high-quality semantically parsed 3D models for dense metropolitan areas is a fundamental urban modeling problem. Although recent advances in acquisition techniques and processing algorithms have resulted in large-scale imagery or 3D polygonal reconstructions, such data-sources are typically noisy, and incomplete, with no semantic structure. In this paper, we present an automatic data fusion technique that produces high-quality structured models of city blocks. From coarse polygonal meshes, street-level imagery, and GIS footprints, we formulate a binary integer program that globally balances sources of error to produce semantically parsed mass models with associated facade elements. We demonstrate our system on four city regions of varying complexity; our examples typically contain densely built urban blocks spanning hundreds of buildings. In our largest example, we produce a structured model of 37 city blocks spanning a total of 1, 011 buildings at a scale and quality previously impossible to achieve automatically.

@article{wrro138594,
volume = {36},
number = {6},
month = {November},
author = {T Kelly and J Femiani and P Wonka and NJ Mitra},
note = {{\copyright} 2017 Copyright held by the owner/author(s). Publication rights licensed to Association for Computing Machinery. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Graphics, https://doi.org/10.1145/3130800.3130823. Uploaded in accordance with the publisher's self-archiving policy.},
title = {BigSUR: large-scale structured urban reconstruction},
publisher = {Association for Computing Machinery},
doi = {10.1145/3130800.3130823},
year = {2017},
journal = {ACM Transactions on Graphics},
url = {http://eprints.whiterose.ac.uk/138594/},
abstract = {The creation of high-quality semantically parsed 3D models for dense metropolitan areas is a fundamental urban modeling problem. Although recent advances in acquisition techniques and processing algorithms have resulted in large-scale imagery or 3D polygonal reconstructions, such data-sources are typically noisy, and incomplete, with no semantic structure. In this paper, we present an automatic data fusion technique that produces high-quality structured models of city blocks. From coarse polygonal meshes, street-level imagery, and GIS footprints, we formulate a binary integer program that globally balances sources of error to produce semantically parsed mass models with associated facade elements. We demonstrate our system on four city regions of varying complexity; our examples typically contain densely built urban blocks spanning hundreds of buildings. In our largest example, we produce a structured model of 37 city blocks spanning a total of 1, 011 buildings at a scale and quality previously impossible to achieve automatically.}
}

T. Kelly and N. Mitra, Simplifying Urban Data Fusion with BigSUR, Architecture_MPS, 2017.

Abstract | Bibtex | Project | PDF

Our ability to understand data has always lagged behind our ability to collect it. This is particularly true in urban environments, where mass data capture is particularly valuable, but the objects captured are more varied, dense, and complex. Captured data has several problems; it is unstructured (we do not know which objects are encoded by the data), contains noise (the scanning process is often inaccurate) and omissions (it is often impossible to scan all of a building). To understand the structure and content of the environment, we must process the unstructured data to a structured form. BigSURi is an urban reconstruction algorithm which fuses GIS (Geographic Information System / mapping) data, photogrammetric meshes, and street level photography, to create clean representative, semantically labelled, geometry. However, we have identified three problems with the system i) the street level photography is often difficult to acquire; ii) novel façade styles often frustrate the detection of windows and doors; iii) the computational requirements of the system are large, processing a large city block can take up to 15 hours. Here we describe the process of simplifying and validating the BigSUR semantic reconstruction system. In particular, the requirement for street level images is removed, and a greedy post-process profile assignment is introduced to accelerate the system. We accomplish this by modifying the binary integer programming (BIP) optimization, and re-evaluating the effects of various parameters. The new variant of the system is evaluated over a variety of urban areas. We objectively measure mean squared error (MSE) terms over the unstructured geometry, showing that BigSUR is able to accurately recover omissions and discard inaccuracies in the input data. Further, we evaluate the ability of the system to label the walls and roofs of input meshes, concluding that our new BigSUR variant achieves highly accurate semantic labelling with shorter computational time and less input data.

@article{wrro141398,
month = {November},
title = {Simplifying Urban Data Fusion with BigSUR},
author = {T Kelly and NJ Mitra},
publisher = {University College London},
year = {2017},
journal = {Architecture\_MPS},
url = {http://eprints.whiterose.ac.uk/141398/},
abstract = {Our ability to understand data has always lagged behind our ability to collect it. This is particularly true in urban environments, where mass data capture is particularly valuable, but the objects captured are more varied, dense, and complex. Captured data has several problems; it is unstructured (we do not know which objects are encoded by the data), contains noise (the scanning process is often inaccurate) and omissions (it is often impossible to scan all of a building). To understand the structure and content of the environment, we must process the unstructured data to a structured form.
BigSURi is an urban reconstruction algorithm which fuses GIS (Geographic Information System / mapping) data, photogrammetric meshes, and street level photography, to create clean representative, semantically labelled, geometry. However, we have identified three problems with the system i) the street level photography is often difficult to acquire; ii) novel fa{\cc}ade styles often frustrate the detection of windows and doors; iii) the computational requirements of the system are large, processing a large city block can take up to 15 hours.
Here we describe the process of simplifying and validating the BigSUR semantic reconstruction system. In particular, the requirement for street level images is removed, and a greedy post-process profile assignment is introduced to accelerate the system. We accomplish this by modifying the binary integer programming (BIP) optimization, and re-evaluating the effects of various parameters.
The new variant of the system is evaluated over a variety of urban areas. We objectively measure mean squared error (MSE) terms over the unstructured geometry, showing that BigSUR is able to accurately recover omissions and discard inaccuracies in the input data. Further, we evaluate the ability of the system to label the walls and roofs of input meshes, concluding that our new BigSUR variant achieves highly accurate semantic labelling with shorter computational time and less input data.}
}

J. Beneš, T. Kelly, F. Děchtěrenko, J. Křivánek, and P. Müller, On Realism of Architectural Procedural Models, Computer Graphics Forum, vol. 36, iss. 2, p. 225–234, 2017.

Abstract | Bibtex | Project | DOI | PDF

The goal of procedural modeling is to generate realistic content. The realism of this content is typically assessed by qualitatively evaluating a small number of results, or, less frequently, by conducting a user study. However, there is a lack of systematic treatment and understanding of what is considered realistic, both in procedural modeling and for images in general. We conduct a user study that primarily investigates the realism of procedurally generated buildings. Specifically, we investigate the role of fine and coarse details, and investigate which other factors contribute to the perception of realism. We find that realism is carried on different scales, and identify other factors that contribute to the realism of procedural and non?procedural buildings.

@article{wrro138592,
volume = {36},
number = {2},
month = {May},
author = {J Bene{\vs} and T Kelly and F D{\ve}cht{\ve}renko and J K{\vr}iv{\'a}nek and P M{\"u}ller},
note = {{\copyright} 2017 The Author(s) Computer Graphics Forum {\copyright} 2017 The Eurographics Association and John Wiley \& Sons Ltd. Published by John Wiley \& Sons Ltd. This is the peer reviewed version of the following article: Bene{\vs}, J. , Kelly, T. , D{\ve}cht{\ve}renko, F. , K{\vr}iv{\'a}nek, J. and M{\"u}ller, P. (2017), On Realism of Architectural Procedural Models. Computer Graphics Forum, 36: 225-234, which has been published in final form at https://doi.org/10.1111/cgf.13121. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving. Uploaded in accordance with the publisher's self-archiving policy.},
title = {On Realism of Architectural Procedural Models},
publisher = {Wiley},
doi = {10.1111/cgf.13121},
year = {2017},
journal = {Computer Graphics Forum},
pages = {225--234},
keywords = {Categories and Subject Descriptors (according to ACM CCS); I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling{--}Geometric algorithms, languages, and systems},
url = {http://eprints.whiterose.ac.uk/138592/},
abstract = {The goal of procedural modeling is to generate realistic content. The realism of this content is typically assessed by qualitatively evaluating a small number of results, or, less frequently, by conducting a user study. However, there is a lack of systematic treatment and understanding of what is considered realistic, both in procedural modeling and for images in general. We conduct a user study that primarily investigates the realism of procedurally generated buildings. Specifically, we investigate the role of fine and coarse details, and investigate which other factors contribute to the perception of realism. We find that realism is carried on different scales, and identify other factors that contribute to the realism of procedural and non?procedural buildings.}
}

T. Kelly, P. Wonka, and P. Mueller, Interactive Dimensioning of Parametric Models, Computer Graphics Forum, vol. 34, iss. 2, p. 117–129, 2015.

Abstract | Bibtex | Project | DOI | PDF

We propose a solution for the dimensioning of parametric and procedural models. Dimensioning has long been a staple of technical drawings, and we present the first solution for interactive dimensioning: a dimension line positioning system that adapts to the view direction, given behavioral properties. After proposing a set of design principles for interactive dimensioning, we describe our solution consisting of the following major components. First, we describe how an author can specify the desired interactive behavior of a dimension line. Second, we propose a novel algorithm to place dimension lines at interactive speeds. Third, we introduce multiple extensions, including chained dimension lines, controls for different parameter types (e.g. discrete choices, angles), and the use of dimension lines for interactive editing. Our results show the use of dimension lines in an interactive parametric modeling environment for architectural, botanical, and mechanical models.

@article{wrro138600,
volume = {34},
number = {2},
month = {May},
author = {T Kelly and P Wonka and P Mueller},
note = {{\copyright} 2015 The Author(s) Computer Graphics Forum {\copyright} 2015 The Eurographics Association and John Wiley \& Sons Ltd. Published by John Wiley \& Sons Ltd.
This is the pre-peer reviewed version of the following article: Kelly, T , Wonka, P and Mueller, P (2015) Interactive Dimensioning of Parametric Models. Computer Graphics Forum, 34 (2). pp. 117-129, which has been published in final form at https://doi.org/10.1111/cgf.12546 This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.
},
title = {Interactive Dimensioning of Parametric Models},
publisher = {Wiley},
doi = {10.1111/cgf.12546},
year = {2015},
journal = {Computer Graphics Forum},
pages = {117--129},
keywords = {Categories and Subject Descriptors (according to ACM CCS); D.2.2 [Computer Graphics]: Design Tools and Techniques{--}User interfaces; I.2.4 [Computer Graphics]: Knowledge Representation Formalisms and Methods{--}Representations (procedural and rule?based)},
url = {http://eprints.whiterose.ac.uk/138600/},
abstract = {We propose a solution for the dimensioning of parametric and procedural models. Dimensioning has long been a staple of technical drawings, and we present the first solution for interactive dimensioning: a dimension line positioning system that adapts to the view direction, given behavioral properties. After proposing a set of design principles for interactive dimensioning, we describe our solution consisting of the following major components. First, we describe how an author can specify the desired interactive behavior of a dimension line. Second, we propose a novel algorithm to place dimension lines at interactive speeds. Third, we introduce multiple extensions, including chained dimension lines, controls for different parameter types (e.g. discrete choices, angles), and the use of dimension lines for interactive editing. Our results show the use of dimension lines in an interactive parametric modeling environment for architectural, botanical, and mechanical models.}
}

T. Kelly, Unwritten procedural modeling with the straight skeleton, University of Glasgow, 2014.

Abstract | Bibtex | Project | PDF

Creating virtual models of urban environments is essential to a disparate range of applications, from geographic information systems to video games. However, the large scale of these environments ensures that manual modeling is an expensive option. Procedural modeling is a automatic alternative that is able to create large cityscapes rapidly, by specifying algorithms that generate streets and buildings. Existing procedural modeling systems rely heavily on programming or scripting – skills which many potential users do not possess. We therefore introduce novel user interface and geometric approaches, particularly generalisations of the straight skeleton, to allow urban procedural modeling without programming. We develop the theory behind the types of degeneracy in the straight skeleton, and introduce a new geometric building block, the mixed weighted straight skeleton. In addition we introduce a simplifcation of the skeleton event, the generalised intersection event. We demonstrate that these skeletons can be applied to two urban procedural modeling systems that do not require the user to write programs. The first application of the skeleton is to the subdivision of city blocks into parcels. We demonstrate how the skeleton can be used to create highly realistic city block subdivisions. The results are shown to be realistic for several measures when compared against the ground truth over several large data sets. The second application of the skeleton is the generation of building’s mass models. Inspired by architect’s use of plan and elevation drawings, we introduce a system that takes a floor plan and set of elevations and extrudes a solid architectural model. We evaluate the interactive and procedural elements of the user interface separately, finding that the system is able to procedurally generate large urban landscapes robustly, as well as model a wide variety of detailed structures.

@unpublished{wrro146627,
editor = {R Poet and P Cockshott},
month = {February},
title = {Unwritten procedural modeling with the straight skeleton},
school = {ARRAY(0x7f6bd8255a08)},
author = {T Kelly},
publisher = {University of Glasgow},
year = {2014},
url = {http://eprints.whiterose.ac.uk/146627/},
abstract = {Creating virtual models of urban environments is essential to a disparate range of applications, from geographic information systems to video games. However, the large scale of these environments ensures that manual modeling is an expensive option. Procedural modeling is a automatic alternative that is able to create large cityscapes rapidly, by specifying algorithms that generate streets and buildings. Existing procedural modeling systems rely heavily on programming or scripting - skills which many potential users do not possess. We therefore introduce novel user interface and geometric approaches, particularly generalisations of the straight skeleton, to allow urban procedural modeling without programming.
We develop the theory behind the types of degeneracy in the straight skeleton, and introduce a new geometric building block, the mixed weighted straight skeleton. In addition we introduce a simplifcation of the skeleton event, the generalised intersection event. We demonstrate that these skeletons can be applied to two urban procedural modeling systems that do not require the user to write programs.
The first application of the skeleton is to the subdivision of city blocks into parcels. We demonstrate how the skeleton can be used to create highly realistic city block subdivisions. The results are shown to be realistic for several measures when compared against the ground truth over several large data sets.
The second application of the skeleton is the generation of building's mass models. Inspired by architect's use of plan and elevation drawings, we introduce a system that takes a floor plan and set of elevations and extrudes a solid architectural model. We evaluate the interactive and procedural elements of the user interface separately, finding that the system is able to procedurally generate large urban landscapes robustly, as well as model a wide variety of detailed structures.}
}

C. Vanegas, T. Kelly, B. Weber, J. Halatsch, D. Aliaga, and P. Müller, Procedural Generation of Parcels in Urban Modeling, Computer Graphics Forum, vol. 31, iss. 2pt3, p. 681–690, 2012.

Abstract | Bibtex | Project | DOI | PDF

We present a method for interactive procedural generation of parcels within the urban modeling pipeline. Our approach performs a partitioning of the interior of city blocks using user?specified subdivision attributes and style parameters. Moreover, our method is both robust and persistent in the sense of being able to map individual parcels from before an edit operation to after an edit operation ? this enables transferring most, if not all, customizations despite small to large?scale interactive editing operations. The guidelines guarantee that the resulting subdivisions are functionally and geometrically plausible for subsequent building modeling and construction. Our results include visual and statistical comparisons that demonstrate how the parcel configurations created by our method can closely resemble those found in real?world cities of a large variety of styles. By directly addressing the block subdivision problem, we intend to increase the editability and realism of the urban modeling pipeline and to become a standard in parcel generation for future urban modeling methods.

@article{wrro138602,
volume = {31},
number = {2pt3},
month = {May},
author = {CA Vanegas and T Kelly and B Weber and J Halatsch and DG Aliaga and P M{\"u}ller},
note = {{\copyright} 2012 The Author(s) Computer Graphics Forum {\copyright} 2012 The Eurographics Association and Blackwell Publishing Ltd. This is the pre-peer reviewed version of the following article: Vanegas, CA, Kelly, T , Weber, B et al. (3 more authors) (2012) Procedural Generation of Parcels in Urban Modeling. Computer Graphics Forum, 31 (2). 2pt3. pp. 681-690, which has been published in final form at https://doi.org/10.1111/j.1467-8659.2012.03047.x. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.},
title = {Procedural Generation of Parcels in Urban Modeling},
publisher = {Wiley},
doi = {10.1111/j.1467-8659.2012.03047.x},
year = {2012},
journal = {Computer Graphics Forum},
pages = {681--690},
keywords = {I.3.5 [Computer Graphics]: Computational Geometry; I.3.6 [Computer Graphics]: Methodology and Techniques},
url = {http://eprints.whiterose.ac.uk/138602/},
abstract = {We present a method for interactive procedural generation of parcels within the urban modeling pipeline. Our approach performs a partitioning of the interior of city blocks using user?specified subdivision attributes and style parameters. Moreover, our method is both robust and persistent in the sense of being able to map individual parcels from before an edit operation to after an edit operation ? this enables transferring most, if not all, customizations despite small to large?scale interactive editing operations. The guidelines guarantee that the resulting subdivisions are functionally and geometrically plausible for subsequent building modeling and construction. Our results include visual and statistical comparisons that demonstrate how the parcel configurations created by our method can closely resemble those found in real?world cities of a large variety of styles. By directly addressing the block subdivision problem, we intend to increase the editability and realism of the urban modeling pipeline and to become a standard in parcel generation for future urban modeling methods.}
}

T. Kelly and P. Wonka, Interactive architectural modeling with procedural extrusions, ACM Transactions on Graphics, vol. 30, iss. 2, 2011.

Abstract | Bibtex | Project | DOI | PDF

We present an interactive procedural modeling system for the exterior of architectural models. Our modeling system is based on procedural extrusions of building footprints. The main novelty of our work is that we can model difficult architectural surfaces in a procedural framework, e.g. curved roofs, overhanging roofs, dormer windows, interior dormer windows, roof constructions with vertical walls, buttresses, chimneys, bay windows, columns, pilasters, and alcoves. We present a user interface to interactively specify procedural extrusions, a sweep plane algorithm to compute a two-manifold architectural surface, and applications to architectural modeling.

@article{wrro138595,
volume = {30},
number = {2},
month = {April},
author = {T Kelly and P Wonka},
note = {(c) 2011, ACM. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Graphics, Vol. 30, No. 2, Article 14, Publication date: April 2011.},
title = {Interactive architectural modeling with procedural extrusions},
publisher = {Association for Computing Machinery},
doi = {10.1145/1944846.1944854},
year = {2011},
journal = {ACM Transactions on Graphics},
keywords = {procedural modeling; roof modeling; urban modeling},
url = {http://eprints.whiterose.ac.uk/138595/},
abstract = {We present an interactive procedural modeling system for the exterior of architectural models. Our modeling system is based on procedural extrusions of building footprints. The main novelty of our work is that we can model difficult architectural surfaces in a procedural framework, e.g. curved roofs, overhanging roofs, dormer windows, interior dormer windows, roof constructions with vertical walls, buttresses, chimneys, bay windows, columns, pilasters, and alcoves. We present a user interface to interactively specify procedural extrusions, a sweep plane algorithm to compute a two-manifold architectural surface, and applications to architectural modeling.}
}