arXiv:2606.08545v1 [cs.CL] 7 Jun 2026
Ishigaki-IDS: An Open-Weight Verifier-Aware Model for Information Delivery Specification Drafting in Building Information Modeling Ryo Kanazawa
Koyo Hidaka
Teppei Miyamoto
Takayuki Kato
ONESTRUCTION Inc. Tottori, Japan
ONESTRUCTION Inc. Tottori, Japan
ONESTRUCTION Inc. Tottori, Japan
ONESTRUCTION Inc. Tottori, Japan
Tomoki Ando
Chenguang Wang
Dayuan Jiang
Naofumi Fujita
ONESTRUCTION Inc. Tottori, Japan
AWS GenAI Innovation Center Tokyo, Japan
AWS GenAI Innovation Center Tokyo, Japan
ONESTRUCTION Inc. Tottori, Japan
Shuhei Saitoh
Atomu Kondo
Koki Arakawa
Daiho Nishioka
ONESTRUCTION Inc. Tottori, Japan
ONESTRUCTION Inc. Tottori, Japan
ONESTRUCTION Inc. Tottori, Japan
ONESTRUCTION Inc. Tottori, Japan
Abstract Building Information Modeling (BIM) projects require information requirements to be described as machine-checkable Information Delivery Specification (IDS) files in order to verify whether building models contain the required attributes. However, IDS authoring remains a practical bottleneck: practitioners must handle domain vocabulary, strict XML schema constraints, and external validator conformance while also checking whether the requirement itself is correctly expressed. We present Ishigaki-IDS, an open-weight LLM specialized for verifier-aware IDS draft generation. The model combines continued pretraining on BIM/IDS corpora, supervised finetuning on information-requirement-to-IDS pairs, and reinforcement learning with verifiable rewards from an external validator. The goal is not to replace expert review, but to move IDS authoring from low-level XML and schema repair toward validator-loadable drafts that practitioners can inspect and correct. On the 166-case expertcreated Ishigaki-IDS-Bench, Ishigaki-IDS-8B1 achieves an IDSAuditPass score of 0.651, a validator-pass metric for generated IDS files, substantially outperforming Claude Opus 4.5, the strongest single-shot LLM baseline we evaluated, at 0.331. It also obtains an Audit-Gated FacetF1 of 0.282, which measures requirement-facet alignment among validator-passing drafts. The same recipe scales: 14B and 32B variants reach IDSAuditPass 0.753 / 0.693 and AuditGated FacetF1 0.392 / 0.369. In a workflow check with six BIM practitioners, Ishigaki-assisted authoring reduced aggregate work time by 54.7% under the same validation and alignment endpoint. These results suggest that verifier-aware IDS generation can reduce the practical burden of converting BIM information requirements into reviewable IDS drafts.
CCS Concepts • Computing methodologies → Natural language generation; • Information systems → Information retrieval; Document representation. 1 https://huggingface.co/ONESTRUCTION/Ishigaki-IDS-8B
Figure 1: Sidewalk IDS generation example, contrasting an incorrect draft with the correct IDS/XML excerpt. The excerpt maps the target to IFCPAVEMENT in IFC4X3_ADD2 and encodes EffectiveWidth as a required property of datatype IFCLENGTHMEASURE with an xs:minInclusive restriction.
Keywords Building Information Modeling, Information Delivery Specification, Industry Foundation Classes, structured generation, domainspecific language models, verifier-aware learning, applied AI
1
Introduction
Construction projects require information requirements shared across design, construction, and maintenance to be described in a structured form that can be inspected and reused in downstream processes. Such information-requirement authoring can be an application target for structured generation with large language models (LLMs), but in practical domains, producing syntactically valid output alone is insufficient. Generated artifacts must follow domain standards, use standard vocabularies correctly, and be inspectable and usable by external tools used in practice. The Building Information Modeling (BIM) information-requirement authoring problem
Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto, Takayuki Kato, Tomoki Ando, Chenguang Wang, Dayuan Jiang, Naofumi Fujita, Shuhei Saitoh, Atomu Kondo, Koki Arakawa, CIKM Applied, Submission Draft, and Daiho Nishioka
studied in this work is a representative example of such constrained structured generation. BIM is the concept of a digital model that handles building geometry together with performance and attribute information, and Industry Foundation Classes (IFC) is a data format that represents building elements and attributes handled in BIM with standardized vocabulary and structure[4, 10]. In contrast, Information Delivery Specification (IDS) is not a format for representing a BIM model itself; it is a standard for describing, in an XML schema, which elements in an IFC-represented model should satisfy which property or value conditions[5]. In practice, information requirements are rarely written as IDS from the beginning. They are often provided as specifications, design notes, or tabular checklists. Generating IDS from such inputs requires interpreting the meaning of the requirement sentence, mapping the target building element to the appropriate IFC vocabulary, selecting the required properties and value conditions, and expressing them as an XML structure conforming to the IDS standard. Moreover, unless the generated IDS passes structure and content checks by an external validator such as IDSAuditTool2 [3], it cannot be loaded into ordinary validation workflows. Since buildingSMART, an international standards organization, standardized IDS 1.0, the foundation for handling BIM information requirements as machine-readable checking specifications has been taking shape. However, creating an IDS specification requires handling IFC vocabulary, IDS/XML structure, validator interpretation, and projectspecific practical requirements at the same time. Therefore, the work of converting specifications or checklists into IDS can become a practical bottleneck in broadly deploying standardized information checking. This work formulates IDS generation as a structured generation task with verifier constraints and presents Ishigaki-IDS-8B, an open-weight model based on Verifier-Aware Multi-Stage Adaptation. The proposed method combines continued pretraining (CPT) on BIM/IFC/IDS-related corpora, supervised fine-tuning (SFT) on requirement-to-IDS pairs, and reinforcement learning with verifiable rewards (RLVR) using IDSAuditTool. The goal is not to replace expert review, but to generate IDS drafts from specifications or checklists that practitioners can audit and edit, thereby reducing the low-level workload of XML authoring, schema conformance, and mapping to standard vocabulary. On the 166-case expert-created Ishigaki-IDS-Bench, Ishigaki-IDS-8B achieves IDSAuditPass=0.651, substantially outperforming Claude Opus 4.5 at 0.331, the strongest model under the evaluated single-shot prompting setting. We also use Audit-Gated FacetF1 (AG-FacetF1) to jointly evaluate validator passing and alignment with required conditions. Furthermore, in a CSV-to-IDS workflow check with six practitioners, observed work time to reach the same validation and alignment endpoint was reduced by 54.7%. Ishigaki-IDS-8B has been publicly available on Hugging Face under a CC BY 4.0 license since March 27, 2026. The contributions of this paper are threefold. • We release Ishigaki-IDS-8B [16], an open-weight IDS drafting model that generates validator-checkable drafts from BIM information requirements for practitioner review. • We present Verifier-Aware Multi-Stage Adaptation for IDS generation, combining BIM/IFC/IDS continued pretraining, 2 https://github.com/buildingSMART/IDS-Audit-tool
requirement-to-IDS supervised fine-tuning, and RLVR with IDSAuditTool and facet-alignment rewards. • We provide applied evidence on Ishigaki-IDS-Bench and a six-practitioner workflow check: Ishigaki-IDS-8B achieves IDSAuditPass 0.651 versus 0.331 for the strongest evaluated single-shot baseline and reduces aggregate time to the fixed endpoint by 54.7%.
2
Related Work
Research on BIM information requirements and IDS has studied how to define information needs as machine-readable checking specifications and connect them to conformance checking over IFC models[4, 5, 10]. Recent work has investigated support for conversion from tabular requirements to IDS[11], IDS-based BIM model enrichment[7], and ontology-based frameworks for IDS evaluation[6], and implementation foundations such as IfcOpenShell and IDSAuditTool have also been developed[3, 14]. In the AEC/BIM domain, LLMs have also been applied to code compliance checking, design support, BIM model generation, and information retrieval[23]; this work focuses on constructing and releasing an open-weight LLM that directly generates IDS drafts capable of passing IDSAuditTool from natural-language or tabular BIM information requirements. IDS generation is also related to semantic parsing[8, 26, 32], but it must simultaneously satisfy IFC vocabulary, the IDS standard, external auditing, and facet-level semantic alignment. Adjacent work on grammar-constrained decoding and structured generation has developed methods that enforce structural validity during generation. Outlines, XGrammar, and vLLM structured outputs provide constrained generation based on regular expressions, JSON schemas, context-free grammars, and related formalisms[9, 29, 30]. These methods can improve XML well-formedness in IDS generation, but they do not solve the problem of combining the building elements, attribute conditions, and value restrictions indicated by the input requirement according to the IDS standard and passing an external validator. Compared with domain-specific LLM studies such as SuperLog[15], Dairy IE[1], LLP-Bench[2], and ecommerce catalog studies[22], Ishigaki-IDS differs in targeting IDS, a construction standard, and placing validator-loadable artifacts at the center of evaluation. From the perspective of model adaptation, this work is related to domain adaptation of open-weight LLMs and optimization based on verifiable rewards. LIMA[33], Tulu 3[20], and Qwen3[31] provide background for constructing domain-specific models, while PRM800K[21], DeepSeekMath[27], and DeepSeek-R1[13] show the effectiveness of optimization using verifiable signals. Ishigaki-IDS applies this line of work to IDS authoring, but its reward is composed not of mathematical correctness checking, but of IDSAuditTool structure/content checks and facet alignment against gold IDS.
3
Task and IDS Representation
Ishigaki-IDS takes as input a BIM information requirement given in natural-language or CSV form, and, when needed, a target IFC version, and outputs a candidate .ids file that practitioners can audit and edit. The task supports the creation of IDS drafts for
Ishigaki-IDS: An Open-Weight Verifier-Aware Model for Information Delivery Specification Drafting in Building Information Modeling
CIKM Applied, Submission Draft,
Figure 2: Verifier-Aware Multi-Stage Adaptation of Ishigaki-IDS. CPT adapts the model to BIM/IFC/IDS vocabulary, SFT teaches requirement-to-IDS mapping, and RLVR updates the model with rewards based on IDSAuditTool and FacetF1. information requirements applied to IFC models; it is not itself conformance checking of submitted IFC models. An IDS consists of one or more specification elements. A specification is one checkable rule in an IDS file and has applicability, which defines the target object, and requirements, which defines the conditions that the target must satisfy. In applicability, the target IFC class is typically specified by entity. Here, an IFC class is a category that represents objects in a BIM model, such as walls, floors, pavements, and equipment elements, as standardized vocabulary. In requirements, attributes, properties, datatypes, value restrictions, and other conditions that the target specified in applicability must satisfy are described. For example, a property requirement expresses which information item the target should have, what type its value should have, and what range or enumeration the value should satisfy. Figure 1 shows an example of mapping an information requirement given in natural language or CSV to target specifications and condition specifications in IDS/XML. The XML in the figure is not a complete IDS file; it excerpts the core ids:entity specification placed in applicability and the core ids:property specification placed in requirements. As this example shows, IDS generation is not merely XML generation. It is a structured generation task under schema constraints in which the target class, property, datatype, and value restriction corresponding to the input requirement are selected over standard vocabularies. In this paper, we use Facet as a semantically meaningful checking unit in IDS. A Facet is not a unit that mechanically counts single IDS/XML elements one by one, but rather a group of target specifications or condition specifications corresponding to an input requirement. For example, an entity specification in applicability is treated as an entity facet that selects the target IFC class. A property requirement in requirements is treated as a property facet that groups the property name, datatype, required/optional cardinality, and value restriction, including the property-set name when
present. The IDS/XML fragment in Figure 1 shows how an entity facet and a property facet appear in IDS. In evaluation, such Facets are used as comparison units. The detailed canonicalization policy and evaluation metrics are defined in Section 5.2.
4
Verifier-Aware Multi-Stage Adaptation
IDS generation must simultaneously satisfy IDS-standard conformance, IFC vocabulary, and facet-level alignment with the input requirement. We use Qwen3 open-weight LLMs as base models and adapt them to IDS generation through Verifier-Aware Multi-Stage Adaptation, consisting of CPT, SFT, and RLVR. Figure 2 shows the pipeline applied to 8B, 14B, and 32B models.
4.1
Adaptation Pipeline
CPT is the stage that moves the Qwen3-base model closer to BIM/IFC/IDS-domain vocabulary and IDS/XML structure. CPT uses a mixed dataset of about 0.5B tokens. The main sources are a technical corpus containing IFC schemas, IDS specifications, and BIM/IFC/IDS-related documents; public IDS examples collected from public repositories after removing overlap with IshigakiIDS-Bench; and a general-domain mix including FineWeb2 [25] to mitigate catastrophic forgetting. The mixing ratio between domain data and general-domain data is approximately 70/30. The objective is to adapt the model distribution to BIM/IFC/IDS terms, IFC entities, IDS rule structure, and XML representation through causal language modeling. SFT is the stage that learns the mapping from BIM information requirements to IDS drafts. The SFT data is built from about 2k seed IDS files curated from public IFC datasets. Each seed IDS is used after confirming that it passes IDSAuditTool. We then extract a facet set from the seed IDS and paraphrase its meaning into a CSV checklist or natural-language specification while preserving the semantics. For paraphrasing, we use multiple open-weight LLMs whose license conditions were checked, and finally construct about
Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto, Takayuki Kato, Tomoki Ando, Chenguang Wang, Dayuan Jiang, Naofumi Fujita, Shuhei Saitoh, Atomu Kondo, Koki Arakawa, CIKM Applied, Submission Draft, and Daiho Nishioka
4k requirement-to-IDS pairs. After synthesis, we filter the data based on facet-set agreement, IDSAuditTool passing of the reconstructed IDS, and XML well-formedness. These paraphrasers are used only as rewriting generators for the SFT data and are not included in the evaluation baselines. RLVR is the stage that further tunes the post-SFT policy using rewards based on an external verifier and facet alignment. For each input 𝑥, we sample four candidate IDS drafts and extract IDS fragments from the candidates. We then run IDSAuditTool structure/content checks and compute FacetF1 against the training reference IDS, as defined in Section 5.2. In RLVR, the policy is updated using a verifier-gated reward that combines the IDSAuditTool pass result and this facet-level agreement. For optimization, we use Group Relative Policy Optimization (GRPO) and add a KL constraint with the SFT model as the reference policy. This encourages generation of IDS drafts that pass the validator and preserve the facets corresponding to the input requirement, rather than merely generating IDS-like XML.
4.2
Training Implementation Details
CPT, SFT, and RLVR were all run with bf16 mixed precision on a two-node configuration with eight NVIDIA H200 GPUs per node, for a total of 16 GPUs. We used the AdamW optimizer for CPT and SFT optimization. In CPT, the 8B, 14B, and 32B models were continually pretrained for 3 epochs with sequence length 4,096, global batch size 768, and learning rate 1 × 10−5 . In SFT, each model was trained for 2 epochs with sequence length 32,768, global batch size 32, and learning rate 5 × 10−6 . For CPT and SFT, we used a warmup-stable-decay (WSD) learning-rate scheduler with warmup, stable, and linear-decay ratios of 0.1, 0.8, and 0.1. In RLVR, each post-SFT model was used as the initial policy and updated with GRPO for 2 epochs. We used the AdamW optimizer for actor updates in GRPO, with training batch size 32, actor learning rate 1 × 10−6 , and fixed KL coefficient 0.001. During rollout, for each input, we generated 𝐾 = 4 candidate responses with temperature 0.6, top-p 0.95, and top-k 20, using maximum prompt length 3,000 tokens and maximum response length 10,000 tokens. These major settings were shared across the 8B, 14B, and 32B models. The training times from CPT through RLVR for the 8B, 14B, and 32B models were approximately 11.2, 21.1, and 42.7 hours, respectively. CPT and SFT were implemented with NVIDIA NeMo Framework[24], and RLVR was implemented as GRPO on verl[28]. Candidate-response generation during RLVR used vLLM[29].
4.3
Verifier-Gated Reward
Let 𝑧 be the raw model output and let 𝑦 = 𝑒 (𝑧) be the extracted IDS. If IDS extraction fails, 𝑒 (𝑧) = ⊥ and the reward is 0. Let 𝑠 (𝑦) and 𝑐 (𝑦) denote the structure check and content check of IDSAuditTool, respectively, and let the audit gate ⊮audit (𝑦) = 1 only when both checks pass. With facet-level F1 against gold IDS 𝑦 ∗ denoted by 𝐹 facet (𝑦, 𝑦 ∗ ), the reward is: 0, 𝑅(𝑧, 𝑦 ) = 0.1 𝑠 (𝑦) + 0.1 𝑐 (𝑦) + 0.8 ⊮audit (𝑦) 𝐹 facet (𝑦, 𝑦 ∗ )
𝑒 (𝑧) = ⊥,
∗
𝑒 (𝑧) = 𝑦.
(1)
This design prevents outputs that violate the IDS standard from receiving high reward solely through facet overlap, and places the main signal on facet alignment after validator passing.
5 Experimental Setup 5.1 Evaluation Benchmark We evaluate IDS generation on Ishigaki-IDS-Bench[17, 18]3 , a curated 166-case evaluation set covering variation in input format, language, target IFC version, and application domain. In this paper, we use Ishigaki-IDS-Bench solely as an evaluation basis; our contribution is Ishigaki-IDS and its verifier-aware adaptation, not benchmark construction. Ishigaki-IDS-Bench is an expert-created benchmark for evaluating IDS generation and contains 108 naturallanguage inputs and 58 CSV inputs, Japanese and English cases, IFC2X3, IFC4, IFC4X3_ADD2, and Architectural, Structural, MEP (Mechanical, Electrical, and Plumbing), and General slices. Each sample has an input requirement, target IFC version, and gold IDS for evaluation. Gold IDS files pass the structure/content checks of IDSAuditTool. Ishigaki-IDS-Bench is an evaluation-only dataset and is not used for CPT/SFT/RLVR, prompt tuning, or model selection. We also confirmed that the gold IDS files in Ishigaki-IDS-Bench are not included in the CPT/SFT/RLVR data as exact text matches or file-hash matches.
5.2
Metrics
As primary metrics, we measure IDSAuditPass, FacetF1, and AGFacetF1. IDSAuditPass is the fraction of cases in which the IDS extracted from the generated output passes both the structure check and the content check of IDSAuditTool. IDS extraction failure, XML parse failure, or failure to pass IDSAuditTool is treated as IDSAuditPass=0. All IDSAuditPass results were computed with IDSAuditTool version 1.0.96. FacetF1 is the F1 between the facet sets extracted from the generated IDS and the gold IDS. Exact string matching would treat XML formatting and semantically unchanged element order as mismatches, so we map facets to canonical tuples containing specification, the applicability/requirements side, facet type, target class, property set, property, datatype, cardinality, value restriction, and related fields before comparison. Differences that can be treated as equivalent under the IDS standard or IFC vocabulary, such as XML formatting, facet order, and value-enumeration order, are absorbed. In contrast, differences that change the meaning or checking result, such as entity, property-set/property, datatype, cardinality, value restriction, letter case in IDS vocabulary, and specification split/merge, are treated as mismatches. IDS extraction failure, XML parse failure, or facet extraction failure gives FacetF1 0. Because target facets and validator results are directly checkable, we do not use LLM-as-judge. Draft usability is measured by IDSAuditPass, and stricter joint quality is measured by AG-FacetF1. AG-FacetF1 activates FacetF1 only for generated IDS files that pass IDSAuditTool, thereby evaluating generation that satisfies both standard conformance and requirement alignment. AG-FacetF1(𝑦, 𝑦 ∗ ) = ⊮audit (𝑦) · 𝐹 facet (𝑦, 𝑦 ∗ ).
(2)
3 Code and evaluation materials are available in the Ishigaki-IDS-Bench repository [19].
Ishigaki-IDS: An Open-Weight Verifier-Aware Model for Information Delivery Specification Drafting in Building Information Modeling
Here, 𝑦 is the generated IDS and 𝑦 ∗ is the gold IDS, and ⊮audit (𝑦) is 1 when the generated IDS passes IDSAuditTool and 0 otherwise. FacetF1 alone is reported as a diagnostic measure of how much content corresponding to the input requirement is recovered independently of audit passing.
5.3
Baselines and Inference
We evaluate Ishigaki-IDS-8B, Ishigaki-IDS-14B, and Ishigaki-IDS32B. Ishigaki-IDS-8B is the released open-weight model artifact, and the 14B/32B models are used as scaling variants. External baselines include representative closed proprietary LLMs (GPT-5.5, Claude Opus 4.5, Gemini 3.1 Pro) and recent open-weight LLMs (gpt-oss120b, Kimi K2.6, DeepSeek V4 Pro, Qwen3.5-397B-A17B). All external baselines were evaluated in May 2026, and the pre-adaptation Qwen3 base models are reported in Table 2. All models receive the same zero-shot IDS-generation system prompt. The prompt includes the target IFC version, the input information requirement, and an instruction to output only IDS. If the output contains a fenced code block, the content inside the block is extracted as IDS; otherwise, the whole output is extracted as IDS. IDS extraction failure, XML parse failure, and max-token truncation are treated as invalid outputs, and we do not perform selection from multiple samples, reranking, post-generation repair, regeneration, or manual correction. Decoding settings are fixed by model family before evaluation and are not tuned on Ishigaki-IDS-Bench. Closed LLMs are evaluated with temperature 0.0, top-p 1.0, and max output tokens 15,000 to the extent accepted by each API. Qwen-family models and Ishigaki variants follow Qwen’s public recommended settings: temperature 0.6, top-p 0.95, top-k 20, min-p 0.0, and max output tokens 15,000. Constrained decoding can improve XML well-formedness, but it is not included in the main baselines because this study evaluates single-shot generation ability that simultaneously satisfies IFC vocabulary selection, IDS-standard conformance, validator passing, and facet alignment with the input requirement.
6
Results and Analysis
Table 1 reports fixed-run single-shot results on Ishigaki-IDS-Bench. Unless otherwise stated, all values are case averages.
6.1
Main Results: Validator-Passing IDS Drafts
Ishigaki-IDS substantially improves validator-passing IDS draft generation. The released 8B model reaches IDSAuditPass 0.651 (108/166), substantially outperforming Claude Opus 4.5 at 0.331 (55/166), the strongest single-shot proprietary baseline. Its AGFacetF1 is also 0.282, exceeding Claude Opus 4.5 at 0.249, and it surpasses the best baseline not only in XML/schema conformance but also in joint quality that includes requirement-facet alignment after audit passing. Table 1 also shows that validator conformance and ungated semantic overlap do not coincide. GPT-5.5 shows FacetF1 0.656, while IDSAuditPass remains 0.277 and AG-FacetF1 is 0.236. Therefore, IDSAuditPass measures the formal endpoint at which a draft can enter a review workflow, and AG-FacetF1 measures the joint quality of standard conformance and requirement alignment.
CIKM Applied, Submission Draft,
The 14B scaling variant shows the strongest overall results, reaching IDSAuditPass 0.753 and AG-FacetF1 0.392. The 32B variant raises ungated FacetF1 to 0.456 and obtains AG-FacetF1 0.369. The practical role of Ishigaki-IDS-8B is to convert requirements into validator-loadable drafts often enough for practitioners to review facets inside an IDS workflow, rather than first repairing XML/schema failures. The semantic review burden remaining after validation is analyzed in Section 6.4.
6.2
Ablation by Adaptation Stage
Table 2 reports the adaptation-stage ablation. We compare Base, CPT+SFT, and +RLVR; SFT denotes supervised fine-tuning after CPT. CPT-only checkpoints produced no valid IDS in 0-shot and 1–3-shot trials under the fixed IDS-only protocol, yielding 0 for all three metrics, so we omit them. The results show that the main gain comes from CPT+SFT, and RLVR further improves IDSAuditPass and AG-FacetF1.
6.3
Practical Scope Across Inputs and Domains
Table 3 reports selected slice results for input format, language, and application domain. The audit-pass rate is relatively stable across input formats and languages, while AG-FacetF1 is higher for CSV inputs and MEP and lower for General, where requirement types are more diverse. In other words, Ishigaki-IDS-8B can generate validator-loadable drafts under multiple conditions, but fine-grained alignment with requirement content depends on input format and domain.
6.4
Failure Analysis and Post-Validation Semantic Residuals
This subsection analyzes IDS-generation failures from two perspectives. The first is structure-level failure that appears as IDS extraction failure, XML parse failure, or failure to pass the structure check of IDSAuditTool, and the second is facet-level semantic mismatch remaining in outputs that pass IDSAuditTool. Generic LLM failures are heavily biased toward structure-level failure. The structure-failure rate is 0.592 for closed proprietary LLMs and 0.747 for general open-weight LLMs, compared with 0.124 for Ishigaki-IDS variants. Closed proprietary LLMs and general open-weight LLMs often generate XML fragments or plausible facet candidates, but frequently fail to generate IDS-standard artifacts that pass audit. Ishigaki-IDS reduces this structure-level failure and shifts the main burden of failure toward facet-level semantic mismatches that practitioners should check after validator passing. Table 4 reports results for the 108 outputs of the released 8B model that passed IDSAuditTool. The median FacetF1 among these audit-passing outputs is 0.40, and 65 cases, or 60.2%, have FacetF1 below 0.5. Therefore, IDSAuditPass should be read not as a metric that guarantees semantic correctness, but as a metric indicating that the output has reached a form that can pass a validator. Audit-passing examples with FacetF1 0 include a case where the gold IDS requires IFCPAVEMENT for a sidewalk requirement but the model outputs IFCSLAB; a case where room-level conditions are required for dwelling-unit area but the output summarizes them into site-level conditions; and a case where, for floor-slope and slip-resistance requirements, the model generates a non-floor
Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto, Takayuki Kato, Tomoki Ando, Chenguang Wang, Dayuan Jiang, Naofumi Fujita, Shuhei Saitoh, Atomu Kondo, Koki Arakawa, CIKM Applied, Submission Draft, and Daiho Nishioka
Table 1: Main results on Ishigaki-IDS-Bench. IDSAuditPass is the validator-pass rate, FacetF1 is diagnostic requirement alignment without the audit gate, and AG-FacetF1 is the joint quality in Eq. (2). Model group
Model
Closed LLM Closed LLM Closed LLM General open-weight LLM General open-weight LLM General open-weight LLM General open-weight LLM Ours (released) Ours (scaling) Ours (scaling)
GPT-5.5 (xhigh) Claude Opus 4.5 Gemini 3.1 Pro gpt-oss-120b (xhigh) Kimi K2.6 DeepSeek V4 Pro Qwen3.5-397B-A17B Ishigaki-IDS-8B Ishigaki-IDS-14B Ishigaki-IDS-32B
Table 2: Ablation by adaptation stage on Ishigaki-IDS-Bench. SFT denotes supervised fine-tuning after CPT. Values are based on single fixed-run outputs.
IDSAuditPass
FacetF1
AG-FacetF1
0.277 0.331 0.259 0.120 0.223 0.163 0.193 0.651 0.753 0.693
0.656 0.422 0.522 0.216 0.484 0.400 0.268 0.359 0.432 0.456
0.236 0.249 0.236 0.111 0.209 0.150 0.164 0.282 0.392 0.369
Table 4: Facet agreement among audit-passing outputs from the released 8B model. 𝑁 = 108. FacetF1 bucket N
Size Stage 8B 8B 8B
IDSAuditPass FacetF1 AG-FacetF1
Base CPT+SFT +RLVR
0.157 0.560 0.651
0.202 0.361 0.359
0.133 0.251 0.282
14B Base 14B CPT+SFT 14B +RLVR
0.139 0.633 0.753
0.213 0.392 0.432
0.117 0.322 0.392
32B Base 32B CPT+SFT 32B +RLVR
0.102 0.627 0.693
0.223 0.464 0.456
0.095 0.359 0.369
1.0 [0.75, 1.0) [0.5, 0.75) (0, 0.5) 0
6.5
Table 3: Selected slice results for the released Ishigaki-IDS-8B. Slice results are descriptive fixed-run case averages. Axis
Slice
N IDSAuditPass AG-FacetF1
Input format CSV 58 Input format Natural language 108 Language English 83 Language Japanese 83 Domain Architectural 48 Domain Structural 42 Domain MEP 40 Domain General 36
0.621 0.667 0.675 0.627 0.688 0.619 0.675 0.611
0.331 0.256 0.305 0.259 0.223 0.300 0.410 0.199
target or an incorrect property. These examples show that IshigakiIDS reduces the XML/IDS-standard failures that were common in existing models, but expert checking is still needed to correctly map requirement content to IDS. Thus, the practical value of IshigakiIDS lies not in eliminating expert review, but in reducing low-level structural repair and providing drafts that allow practitioners to focus on checking IDS content.
14 20 9 40 25
% Review implication 13.0 18.5 8.3 37.0 23.1
Matches the gold facet set under the scorer Minor or local facet checks are needed Several facets require checking Major facet correction is needed Passes audit but does not align with gold
Practical Workflow Check with BIM/IDS Practitioners
The previous analysis showed that Ishigaki-IDS makes it easier to generate validator-loadable IDS drafts, while its outputs do not become professionally correct IDS as-is. Therefore, the practical question in this work is not whether expert review can be replaced, but how much the burden of initial IDS construction and XML/schema repair required before review can be reduced. To check this point, we conducted a small exploratory workflow check with six external AEC/BIM practitioners who were neither authors nor internal ONESTRUCTION members. Participants belonged to general construction, design, and MEP-related companies and had roles such as BIM promotion and BIM management, design quality control, technical research, and DX promotion. All six participants had BIM work experience; three had IDS authoring experience, and three did not. Some participants had experience in buildingSMART-related activities, but this exercise was not a buildingSMART validation. No compensation or incentive was provided for participation. Participants consented to the timed exercise and anonymized reporting of aggregate and per-participant timing results; no personally identifiable information is reported. The task in this experiment was to convert tabular BIM information requirements into IDS. We prepared two corresponding CSVto-IDS tasks derived from the publicly available Forestry Agency’s Wooden BIM Model Parameter Guide[12], with comparable input format, target IFC vocabulary, and gold IDS scale. One task handled design and energy-related information requirements, and the other
Ishigaki-IDS: An Open-Weight Verifier-Aware Model for Information Delivery Specification Drafting in Building Information Modeling
Table 5: Practitioner workflow-check times (s) to the fixed endpoint: IDSAuditPass and FacetF1=1.0; one-sided sign test 𝑝 ≈ 0.016. Participant OpenAEC Ishigaki-assisted Reduction P1 P2 P3 P4 P5 P6
420 450 606 644 660 585
180 120 324 284 286 330
57.1% 73.3% 46.5% 55.9% 56.7% 43.6%
Median Aggregate
595.5 3365
285.0 1524
56.3% 54.7%
handled envelope area, spatial attributes, dimensions, and identifiers. Each participant completed one task with the OpenAEC4 editor baseline and the other task with the Ishigaki-assisted workflow. Here, OpenAEC was used as a manual IDS editor environment, not as a generative IDS model. In the baseline, participants manually created and corrected IDS in OpenAEC. In the assisted workflow, Ishigaki-IDS-8B first generated an IDS draft from the CSV input, and participants then checked, corrected, validated, and re-corrected the draft in OpenAEC. Regarding OpenAEC, four participants used it for the first time in this exercise, and only two had prior experience using it. The evaluation endpoint was fixed across the two workflows. The submitted IDS had to pass IDSAuditTool and reach FacetF1=1.0 against a preconstructed gold IDS. During the exercise, participants did not see the gold IDS or facet-level gold annotations. After each submitted candidate, an evaluator ran the fixed completion check and returned only whether the criterion had been met. No gold IDS, missing facet, or target facet information was revealed during the task. The FacetF1 judgment used the same canonicalization policy as Section 5.2, and gold IDS was not shown to participants. Time included the sequence of operations required to satisfy the endpoint. All six participants satisfied the endpoint faster with Ishigaki assistance (Table 5). Aggregate time decreased from 3365 seconds for the OpenAEC baseline to 1524 seconds for the Ishigaki-assisted workflow, corresponding to a 54.7% reduction. Participant-level reductions ranged from 43.6% to 73.3%, and median time decreased from 595.5 seconds to 285.0 seconds. However, this result is not evidence of generalized production productivity, and should be positioned as a directional workflow check. In other words, IshigakiIDS can reduce the effort required for initial IDS construction and low-level schema repair while retaining expert review. The released artifact ONESTRUCTION/Ishigaki-IDS-8B has been available on Hugging Face under a CC BY 4.0 license since March 27, 2026[16]. As of May 20, 2026, it had recorded 98 downloads. In the future, we plan to integrate it into an ONESTRUCTION product and deploy it as a feature that supports IDS draft generation by actual end users. 4 https://openaec.jp/en
7
CIKM Applied, Submission Draft,
Limitations and Scope
Ishigaki-IDS-Bench is a benchmark for diagnosing IDS draftgeneration ability. It contains curated requirements derived from real-world use cases, but it does not represent all AEC organizations, project types, or IDS authoring practices. In addition, because FacetF1 and AG-FacetF1 depend on the annotation and normalization policies of the benchmark, they do not cover all valid IDS encodings and should be interpreted as policy-based alignment scores. The main model evaluation uses a zero-shot single-shot generation protocol and, except for the small workflow check with OpenAEC in Section 6.5, does not compare against broad end-toend authoring workflows that combine multiple-sample generation, validator-feedback repair, constrained decoding, retrieval, manual correction, or commercial IDS tools. Also, due to evaluation-budget constraints, we do not report variance across multiple seeds. The practitioner exercise is a controlled check based on six participants and two tasks, so it should be read as directional evidence under the same endpoint, and it does not show that the same effect would hold for other IDS editors, large-scale requirements, workflows spanning multiple organizations, or production use, nor does it demonstrate generalized productivity improvement. Because task assignment and workflow order were not fully counterbalanced, part of the observed work-time reduction may also reflect task familiarity or order effects.
8
Conclusion
This work formulated the problem of generating IDS drafts from BIM information requirements as verifier-constrained structured generation that must simultaneously satisfy IDS-standard conformance and facet-level alignment, and constructed and released the open-weight model Ishigaki-IDS through Verifier-Aware MultiStage Adaptation. On Ishigaki-IDS-Bench, the released 8B model achieved IDSAuditPass 0.651, exceeding 0.331 for the best nonIshigaki single-shot baseline, and in a practitioner check with six participants, work time was reduced by 54.7% under the same endpoint. At the same time, semantic residuals remain after audit passing, so the contribution of this work is not autonomous IDS authoring, but the release of an open-weight IDS draft generator that can hand off drafts to expert review.
Acknowledgement This work was conducted as part of the GENIAC (Generative AI Accelerator Challenge) Project, which aims to strengthen Japan’s capability to develop generative AI and is promoted by the Ministry of Economy, Trade and Industry (METI) and the New Energy and Industrial Technology Development Organization (NEDO).
GenAI Usage Disclosure Generative AI tools were used as part of the research subject and experimental procedures. This includes construction of Ishigaki-IDS variants, synthesis of requirement-to-IDS pairs, and comparative evaluation with baseline models. During manuscript preparation, generative AI was used for language polishing, terminology consistency checking, LaTeX formatting support, and checking Japanese and English expressions. The authors checked the evaluation data,
Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto, Takayuki Kato, Tomoki Ando, Chenguang Wang, Dayuan Jiang, Naofumi Fujita, Shuhei Saitoh, Atomu Kondo, Koki Arakawa, CIKM Applied, Submission Draft, and Daiho Nishioka
gold IDS, metric computation, and result interpretation, and revised the outputs. The authors are responsible for all claims, experimental design, analysis, conclusions, and the final manuscript.
References [1] Chunyan An, Yuying Huang, Qiang Yang, Siyu Yuan, and Zhixu Li. 2025. LLMPowered Information Extraction for the Dairy Financial Domain: Tackling Data Scarcity and Ambiguity. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, New York, NY, USA, 55–64. doi:10.1145/3746252.3761030 [2] Anand Brahmbhatt, Mohith Pokala, Rishi Saket, and Aravindan Raghuveer. 2024. LLP-Bench: A Large Scale Tabular Benchmark for Learning from Label Proportions. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, New York, NY, USA, 4374–4381. doi:10.1145/3627673.3680032 [3] buildingSMART International. 2024. IDSAuditTool. https://github.com/ buildingSMART/IDS-Audit-tool. Official buildingSMART tool for auditing IDS files. Accessed: 2026-05-17. [4] buildingSMART International. 2026. Industry Foundation Classes (IFC). https://www.buildingsmart.org/standards/bsi-standards/industry-foundationclasses/. Accessed: 2026-05-08. [5] buildingSMART International. 2026. Information Delivery Specification (IDS). https://www.buildingsmart.org/standards/bsi-standards/informationdelivery-specification-ids/. Accessed: 2026-05-08. [6] Tomo Cerovšek and Mohamed Omar. 2025. Advancing Semantic Enrichment Compliance in BIM: An Ontology-Based Framework and IDS Evaluation. Buildings 15, 15 (2025), 2621. doi:10.3390/buildings15152621 [7] Giancarlo de Marco, Cinzia Slongo, and Dietmar Siegele. 2024. Enriching Building Information Modeling Models through Information Delivery Specification. Buildings 14, 7 (2024), 2206. doi:10.3390/buildings14072206 [8] Li Dong and Mirella Lapata. 2016. Language to Logical Form with Neural Attention. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Berlin, Germany, 33–43. doi:10.18653/v1/P16-1004 [9] Yixin Dong, Charlie F. Ruan, Yaxing Cai, Ruihang Lai, Ziyi Xu, Yilong Zhao, and Tianqi Chen. 2024. XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models. arXiv:2411.15100. doi:10.48550/arXiv.2411. 15100 [10] Chuck Eastman, Paul Teicholz, Rafael Sacks, and Ghang Lee. 2018. BIM Handbook: A Guide to Building Information Modeling for Owners, Designers, Engineers, Contractors, and Facility Managers (3 ed.). Wiley, Hoboken, NJ, USA. [11] Simon Fischer, Harald Urban, Christian Schranz, and Gerhard Zucker. 2025. Bridging the Gap Between Tabular Information Requirements and the Information Delivery Specification (IDS). Buildings 15, 7 (2025), 1017. doi:10.3390/ buildings15071017 [12] Forestry Agency, Ministry of Agriculture, Forestry and Fisheries of Japan. 2025. Wooden BIM Model Parameter Guide: Explanation of How to Assign Information to Object Data in Wooden Design BIM Models. https://www.rinya.maff.go.jp/j/ mokusan/attach/pdf/bim-19.pdf. In Japanese. Accessed: 2026-05-17. [13] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, et al. 2025. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature 645 (2025), 633–638. doi:10.1038/s41586-02509422-z [14] IfcOpenShell Contributors. 2026. IfcOpenShell. https://docs.ifcopenshell.org/ ifcopenshell.html. Open source IFC library and geometry engine. Accessed: 2026-05-17. [15] Yuhe Ji, Yilun Liu, Feiyu Yao, Minggui He, Shimin Tao, Xiaofeng Zhao, Chang Su, Xinhua Yang, Weibin Meng, Yuming Xie, Boxing Chen, Shenglin Zhang, and Yongqian Sun. 2025. Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, New York, NY, USA, 1135–1144. doi:10.1145/3746252.3761189 [16] Ryo Kanazawa, Koyo Hidaka, Tomoki Ando, Chenguang Wang, Dayuan Jiang, and Daiho Nishioka. 2026. ONESTRUCTION/Ishigaki-IDS-8B. https://huggingface. co/ONESTRUCTION/Ishigaki-IDS-8B. doi:10.57967/hf/8871 Model card and weights. Accessed: 2026-05-20. [17] Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto, Takayuki Kato, Tomoki Ando, Chenguang Wang, Dayuan Jiang, Naofumi Fujita, Shuhei Saitoh, Atomu Kondo, Koki Arakawa, and Daiho Nishioka. 2026. Ishigaki-IDS-Bench. doi:10.57967/hf/ 8873 Hugging Face dataset. Accessed: 2026-05-21. [18] Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto, Takayuki Kato, Tomoki Ando, Chenguang Wang, Dayuan Jiang, Naofumi Fujita, Shuhei Saitoh, Atomu Kondo, Koki Arakawa, and Daiho Nishioka. 2026. Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements. arXiv:2605.22079 [cs.CL] https://arxiv.org/abs/2605.22079
[19] Ryo Kanazawa, Koyo Hidaka, Teppei Miyamoto, Takayuki Kato, Tomoki Ando, Chenguang Wang, Dayuan Jiang, Naofumi Fujita, Shuhei Saitoh, Atomu Kondo, Koki Arakawa, and Daiho Nishioka. 2026. Ishigaki-IDS-Bench: Evaluation Code and Reproducibility Repository. doi:10.5281/zenodo.20362616 GitHub repository release v1.0.0. Accessed: 2026-05-24. [20] Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James Validad Miranda, Alisa Liu, Nouha Dziri, Xinxi Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Christopher Wilhelm, Luca Soldaini, Noah A. Smith, et al. 2025. Tulu 3: Pushing Frontiers in Open Language Model Post-Training. COLM 2025, OpenReview. https://openreview.net/forum?id=i1uGbfHHpH [21] Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s Verify Step by Step. arXiv:2305.20050. doi:10.48550/arXiv.2305.20050 [22] Gang Luo, Julien Han, Hayreddin Ceker, and Karim Bouyarmane. 2025. Using Large Language Models to Improve Product Information in E-commerce Catalogs. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, New York, NY, USA, 6873–6874. doi:10.1145/3746252.3761437 [23] Soumya Madireddy, Lu Gao, Zia Ud Din, Kinam Kim, Ahmed Senouci, Zhe Han, and Yunpeng Zhang. 2025. Large Language Model-Driven Code Compliance Checking in Building Information Modeling. Electronics 14, 11 (2025), 2146. doi:10.3390/electronics14112146 [24] NVIDIA Corporation. 2026. NVIDIA NeMo Framework. https://docs.nvidia.com/ nemo-framework/user-guide/latest/overview.html. Documentation. Accessed: 2026-05-21. [25] Guilherme Penedo, Hynek Kydlíček, Vinko Sabolčec, Bettina Messmer, Negar Foroutan, Amir Hossein Kargaran, Colin Raffel, Martin Jaggi, Leandro Von Werra, and Thomas Wolf. 2025. FineWeb2: One Pipeline to Scale Them All – Adapting Pre-Training Data Processing to Every Language. arXiv:2506.20920. doi:10.48550/ arXiv.2506.20920 [26] Maxim Rabinovich, Mitchell Stern, and Dan Klein. 2017. Abstract Syntax Networks for Code Generation and Semantic Parsing. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Vancouver, Canada, 1139–1149. doi:10.18653/v1/P17-1105 [27] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300. doi:10.48550/arXiv.2402.03300 [28] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. HybridFlow: A Flexible and Efficient RLHF Framework. arXiv:2409.19256. doi:10.48550/arXiv.2409.19256 [29] vLLM Team. 2026. Structured Outputs. https://docs.vllm.ai/usage/structured_ outputs.html. vLLM documentation. Accessed: 2026-05-17. [30] Brandon T. Willard and Rémi Louf. 2023. Efficient Guided Generation for Large Language Models. arXiv:2307.09702. doi:10.48550/arXiv.2307.09702 [31] An Yang et al. 2025. Qwen3 Technical Report. arXiv:2505.09388. doi:10.48550/ arXiv.2505.09388 [32] Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2018. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, 3911–3921. doi:10.18653/v1/D181425 [33] Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023. LIMA: Less Is More for Alignment. arXiv:2305.11206. doi:10.48550/arXiv.2305.11206