Prism: A Creative Assistant for Architectural Education

What
The goal is to advance an AI‑based model that enables the identification and labelling of architectural elements from our 100,000 scanned (and growing number of) hand drawings. The labelled images form the basis of a structured database from which students can search for architecture‑specific terms (e.g. constructive principles or spatial effects) via a user interface using text and image input.
The output will be a list of projects with drawings containing similar elements. Prism includes both the database and the user interface.

Why
The development of architecture‑specific competencies requires reference‑based learning. With the growing number of students, a streamlined academic calendar, and an increasing amount of references, there is a need for time‑ and place‑independent access to these materials, enabling self‑paced learning.
Learning emerges through increased comparison, iterative feedback, and reflection. This process strengthens students’ intuition in solving design tasks and helps preserve high teaching quality.

How
The vision‑language model BLIP (Junnan Li, 2022) is suitable for analysing and labelling hand drawings but has limited capabilities with regard to architectural concepts. We will extend the model through the following steps:

• Labelling 1,000 scanned hand drawings according to architecture‑specific terms (a data expert will be hired for this task)
• Training the AI‑based model with the labelled images (data expert)
• Applying the model to all images to assign labels and integrate them into the database (Prism)

Students

• Time‑ and place‑independent access to an extensive database of labelled drawings from past projects for comprehensive reference analysis, self‑reflection, and design iterations, strengthening intuition for solving more complex design tasks
• Enhanced mutual and self‑learning through structured access to reference projects, comparative analysis, and feedback integration
• Increased knowledge base
• Greater confidence in expressing ideas and concepts in a fast and abstract way, while remaining understandable to all parties along the value chain

Lecturers

• Structured access to past student work for lectures and assessment within D‑ARCH
• Adaptable model that can be extended to or translated into other courses, chairs, teaching formats, and even disciplines; relevant for focus projects (PAKETH) and supportive of appropriate management to reduce student workload through mutual learning and aligned expectations
• Reduced workload for assistants coaching students, enabled by increased self‑learning

Department / ETH

• Database and model can be opened to other chairs within D‑ARCH and to other schools, increasing both data and user base. This supports the development of students’ architectural intuition and may foster more context‑aware (e.g. location‑sensitive) architecture rather than the uniform designs often visible today
• Model can also be applied to other disciplines, such as engineering, where sketches play a key role in solving technical challenges; field‑specific adaptation and training would be required

Starting point of this project is the AI-based model BLIP that has the capability to identify objects from hand drawings and sketches. However, the model has limited capabilities and cannot identify architecture- and construction specific elements, e.g. structural, building envelope, spatial effects and atmospheres, perspectives and styles. Therefore, the goals of the project are: Define the comprehensive assistant specifications (input, output) Labelling of drawings from the archive according architecture-specific terms, e.g. structural, building envelope, spatial effects, perspectives and drawing styles Train the model according to specification Validation of the model Analysis and labelling of more than 100.000 drawings Create database and user-interface Input/output: through the user interface, the database can be searched for specific terms, which then displays the identified projects Setup of the required infrastructure to host the database, and provide the web-access.