Kale: A Transformation-Safe Spreadsheet System

# Kale: A Transformation-Safe Spreadsheet System
Source: [https://arxiv.org/html/2608.26345](https://arxiv.org/html/2608.26345)
Jacob YimEmail:[j1yim@ucsd\.edu](mailto:[email protected])Affiliation:University of California, San DiegoAjinkyaBokadeEmail:[acbokade@gmail\.com](mailto:[email protected])Affiliation:Salesforce, IncMounikaPadalaEmail:[mounikapadala11@gmail\.com](mailto:[email protected])Affiliation:Apple, IncJuliaEpshteinEmail:[jepshtein@umass\.edu](mailto:[email protected])Affiliation:University of Massachusetts, AmherstPriyankaBhatiaEmail:[b\.priyanka0210@gmail\.com](mailto:[email protected])Affiliation:Meta, IncPiyushChauhanEmail:[piyushchauhan1004@gmail\.com](mailto:[email protected])Affiliation:Eukarya, IncSimranGillEmail:[gillsimu98@gmail\.com](mailto:[email protected])Affiliation:Apple, IncAniketGuptaEmail:[ang042@ucsd\.edu](mailto:[email protected])Affiliation:University of California, San DiegoGrishmaGurbaniEmail:[grishmagurbani1998@gmail\.com](mailto:[email protected])Affiliation:University of California, San DiegoVaibhavKhetanEmail:[vaibhavkhetan2703@gmail\.com](mailto:[email protected])Affiliation:Apple, IncArushiMunjalEmail:[amunjal@ucsd\.edu](mailto:[email protected])Affiliation:University of California, San DiegoJefferyTungEmail:[j7tung@ucsd\.edu](mailto:[email protected])Affiliation:University of California, San DiegoJoannaYangEmail:[joy002@ucsd\.edu](mailto:[email protected])Affiliation:University of California, San Diego

###### Abstract

Spreadsheet formulas can refer to rectangular ranges of arbitrary size\. When a user changes the structure of a referenced table, the spreadsheet system updates the references to refer to a new range\. Unfortunately, this new range may differ from the user’s expectations, introducing bugs in spreadsheets\. We describe a user study showing that standard reference semantics are error\-prone, resulting in significant risk to users\. We introduce Kale, a prototype system that eliminates the risk of inserting these kinds of bugs by restricting the kinds of references that can be expressed\. We show that Kale can be used effectively by users to complete tasks that are error\-prone in traditional spreadsheet systems\. Finally, we describe a corpus study that evaluates the extent to which the reference restrictions in Kale might have implications on users\.

###### keywords

spreadsheets, spreadsheet errors, end\-user programming

## 1Introduction

Spreadsheets may well represent the world’s most popular programming language[Scaffidi et al\. \(2005\)](https://arxiv.org/html/2608.26345#bib.bib1), with likely hundreds of millions of users worldwide[Gislason \(2018\)](https://arxiv.org/html/2608.26345#bib.bib2)\. They are used for essential work in nearly every industry, including business, finance, economics, and science\. Unfortunately, many spreadsheets are buggy, potentially leading users to incorrect conclusions\. Various studies have shown that about 95% of spreadsheets contain errors[Panko \(2015\)](https://arxiv.org/html/2608.26345#bib.bib4)\. Spreadsheet errors have resulted in significant economic and scientific costs\. For example, J\.P Morgan Chase’s 2012 “London Whale” debacle stemmed from a spreadsheet error, costing the company $6\.2B[JPMorgan Chase & Co\. \(2013\)](https://arxiv.org/html/2608.26345#bib.bib5)\. Approximately one fifth of genetics papers that included supplementary Excel gene lists were found to include erroneous gene name conversions[Ziemann et al\. \(2016\)](https://arxiv.org/html/2608.26345#bib.bib6)\.

Most prior work on spreadsheet errors has focused on the possibility of entering incorrect formulas\. In this work, we provide empirical evidence for a*different*cause of spreadsheet errors that has not, as far as we are aware, been studied previously\. When a spreadsheet table that is referenced by formulas is modified structurally \(for example, by inserting a row\), the referencing formulas are automatically updated by the spreadsheet system\. For example, if a formula references cell B2 of a table, and the user inserts a row at the top of the table, the formula is then updated to refer to cell B3\. This behavior is intended to preserve the semantics of the formula, since any data that was previously located in cell B2 will now be located in cell B3\.

But do spreadsheet references actually refer to the*data*in a cell, or do they refer to the*geometry*of the cell? In the previous example, they refer to the data; if they referred to the geometry, they would still point to the same coordinates after the change\. Consider the formulaSUM\(B2:C3\), which refers to a 2x2 rectangle of cells, and suppose that the user drags and drops cell C3 to a new location\. The formula remains unchanged, but it may evaluate to a different value because cell C3 is now empty\. In this case, the reference apparently referred to the*geometry*, not the data\. We refer to this problem as*reference instability*: references are unstable under structural changes to spreadsheets, which can result in the introduction of bugs into previously\-correct spreadsheets\.

![Refer to caption](https://arxiv.org/html/2608.26345v1/t3kale.png)Figure 1:Kale, as seen in our Payroll task\. A spreadsheet table is shown at the bottom; above are tabs showing available sheets and a set of buttons with available actions\.Kale, shown in[Figure1](https://arxiv.org/html/2608.26345#S1.F1), is a novel prototype spreadsheet system that improves safety by re\-envisioning the syntax and semantics of spreadsheet references\. In Kale, references can refer to individual cells, whole rows, and whole columns, but not spans of rows or columns, and not arbitrary rectangular ranges\. In doing so, Kale avoids the reference instability that is fundamental to traditional spreadsheet systems\.

Kale provides a key safety property not provided by traditional spreadsheets:*preservation of referenced data through structural transformations*\. That is, when rows or columns are inserted, removed, or permuted, formulas that reference data in affected cells will continue to refer to the same data after the transformation\. Furthermore, when a formula itself is moved, it will refer to either the same data as before or the same relative offset according to the intent of the formula’s author\. This property does not hold in traditional spreadsheets\.

Kale is designed to eliminate the risks posed by traditional spreadsheet systems: transformations can introduce bugs in formulas by breaking assumptions made by formula authors about which data formulas refer to\. Each transformation presents a risk, which we indexR1,R2, etc\.

R1: Insertions adjacent to ranges are ambiguous\. When rows or columns are inserted immediately above or below a referenced range, it can be unclear whether the reference should be updated to include the new rows or columns\. If the new row or column represents another instance that is similar to the nearby ones, failing to include the new item represents an error\. Sheets, Excel, and Numbers do not extend references to include new adjacent rows, but Excel emits a warning in the referencing cell\.

R2: Sorting permutes referenced data\. Sorting permutes rows in a table without updating any references to the sorted table\. If an author intended a formula to refer to data in a row that the sort operation moves, then sorting breaks that formula\.

R3: Moving cells changes references\. Moving rows or columns causes references that start or end at the moved row or column to be updated\. For example, if a formula referencesB2:B3and the user moves row 3 below row 4, then the formula now refers toB2:B5, suddenly including the data in the row that was previously at index 4\. In Excel, this operation is triggered by cut/paste; in Numbers and Sheets, by drag/drop\.

R4: Names do not always move alongside data\. Cells can be given names\. When structural changes occur, it can be unclear whether names will move\. In Excel and Sheets, names move with values when cells are cut and pasted but not when cells are sorted\.

R5: References are relative by default\. When a formula cell is copy\-pasted or drag\-filled, any resulting formulas are automatically adjusted to their new location\. For instance, if a formula in cell B1 referencing cell A1 \(the cell directly to the left\) is drag\-filled down to cell B2, the new formula now references cell A2\. If the author intends this new formula to reference the*same*cell as the copied formula, this behavior creates an error\. A scenario demonstrating a variation of this error is shown in[Figure2](https://arxiv.org/html/2608.26345#S1.F2)\. To avoid this error, the author would have to explicitly write absolute references; in Excel, Sheets, and Numbers, this is done by using the dollar sign symbol \($\) to mark absolute row and column references \(e\.g\.$A$1\)\.

![Refer to caption](https://arxiv.org/html/2608.26345v1/dragfillex1.png)\(a\)The user uses the exchange rate in cell A2 to convert the value in cell B2 to CAD\.
![Refer to caption](https://arxiv.org/html/2608.26345v1/dragfillex2.png)\(b\)The user drag\-fills the formula down\.
![Refer to caption](https://arxiv.org/html/2608.26345v1/dragfillex3.png)\(c\)The value of the produced formula is unexpectedly 0\.
![Refer to caption](https://arxiv.org/html/2608.26345v1/dragfillex4.png)\(d\)Upon closer inspection, the user sees that the new formula no longer references the intended exchange rate\.

Figure 2:References are relative by default in Excel, leading to errors when users intend to use absolute references\.Confusingly, although traditional spreadsheets distinguish between*relative*and*absolute*references, this distinction does not address risks R1\-R4\. In Excel, Sheets, and Numbers, the absolute vs\. relative distinction*only*affects which cells are referenced by a formula that is created by copy/paste or drag\-fill; it has no influence on how a formula is updated when the structure of the referenced table is modified\. For example, if a formula includes the absolute reference$B$2and the formula is copy/pasted one cell below its original location, the formula would still refer to$B$2\. In contrast, if the reference had been relative \(B2\), it would then refer toB3\. But both absolute and relative references are treated identically when row 2 is drag/dropped to a new location: the original formula might then refer to$B$7, for example, if the new location of the data is row 7\. In Kale, the terms*absolute*and*relative*are re\-defined so that they capture the semantics that a reference should have when a table is re\-structured\.

We describe a user study comparing Kale to traditional spreadsheets\. We asked participants to complete four tasks using either Google Sheets or Kale\. Each task is associated with a subset of the risks described in Section 1\. The study focuses on assessing whether, when participants are given tasks that may break formulas, participants indeed transform their document in ways that introduce bugs\. For participants who do introduce bugs, we assess whether they notice and are able to successfully fix them\.

Because Kale restricts the kinds of references that can be expressed, there is a possibility that common problems that spreadsheets are used for cannot be solved in Kale\. We describe an evaluation of the impact of Kale’s restrictions by manually porting 60 randomly\-selected spreadsheets from the EUSES spreadsheet corpus[Fisher and Rothermel \(2005\)](https://arxiv.org/html/2608.26345#bib.bib8)\. We successfully translated all of the spreadsheets to use Kale’s reference approach\. Nonetheless, due to the restrictions, we regard Kale as a design exploration that shows the potential benefits of controlling unsafe references; it is possible that future systems that relax some of these restrictions could be more usable in practice\.

This paper describes the following contributions:

1. 1\.A new spreadsheet system, Kale, which prevents users from introducing the aforementioned bugs with new syntax and semantics for spreadsheet reference \([Section2](https://arxiv.org/html/2608.26345#S2)\)\.
2. 2\.A user study \([Section4](https://arxiv.org/html/2608.26345#S4)\) showing that when users of a traditional spreadsheet, Google Sheets, re\-structure documents, they often introduce latent spreadsheet bugs\. In contrast, users are more often able to complete those tasks successfully using Kale\.
3. 3\.A corpus study showing that most existing spreadsheets can be represented in Kale \([Section5](https://arxiv.org/html/2608.26345#S5)\)\.

## 2The Kale Spreadsheet System

### 2\.1Design

The design of Kale prevents errors that arise from reference instabilities caused by structural changes to spreadsheets\. Unlike traditional spreadsheets, Kale does not allow references to arbitrary rectangular ranges\. Instead, references can refer to individual cells, whole columns, or whole rows\.

Unlike traditional spreadsheet systems, which have extremely large grids, Kale sheets are organized into*tables*, akin to tables in Apple’s Numbers spreadsheet system\. In the rest of this paper, the word*table*will be used to refer to a grid of cells, regardless of whether that grid has configurable \(like Numbers\) or extremely large \(like Sheets or Excel\) extent\. In Kale, tables have a header row that defines names of columns\. Row indices start at 0, and row 0 is the row*below*the header row\. The header row cannot be referenced\.[Figure1](https://arxiv.org/html/2608.26345#S1.F1)shows an example from the Payroll task in our evaluation\.

Kale supports the following reference types; examples assume that there is a column calledColthat can be referenced\.

- •Single\-cell references refer to the content in the referenced cell and are updated to refer to new locations when the cell is moved\. These can be absolute \(e\.g\.,Col\[0\]refers to the 0th index in the Col column\) or relative \(e\.g\.,Col\[\+1\]refers to the cell in the Col column that is one row below the referencing formula\)\.
- •Column references refer to all non\-header cells in the referenced column\. For example,SUM\(Col\)sums all non\-header cells in the Col column\.
- •Row references refer to all cells in the referenced row:SUM\(\[0\]\)sums all cells in the row at index 0\.

Kale’s new definition of*absolute*and*relative*enables users to encode their intent when writing a formula\. If a formula refers to a row that is moved \(because of a row insertion, deletion, a sort operation, or a drag/drop operation\), the new referenced row depends on whether the reference is absolute or relative\. If a reference is absolute \(e\.g\.,Col\[0\]\), and the referenced row is moved, then afterward, the reference points to the new location of the moved row\. If relative \(e\.g\.,Col\[\+1\]\), the reference then points to the same relative offset as before, referring to different data\. Absolute references are represented in terms of unique IDs, which Kale assigns to each row of each table\. As a result, when rows are inserted, deleted, or permuted, formulas that use absolute references continue to refer to the same rows as they did before the changes\.

Kale’s formula syntax is traditional in that it supports infix arithmetic operators and function calls with parenthesized lists of comma\-separated arguments\. Because of the novelty of the reference syntax and semantics, we show the reference syntax in[Figure3](https://arxiv.org/html/2608.26345#S2.F3)\. The reference syntax is designed to maximize expressiveness\. In addition to supporting relative and absolute row offsets that are expressed as literal numbers, Kale also supports relative offsets that are arbitrary expressions\. For example,Col\[\+1\+1\]is a relative reference to the column calledColtwo rows below the current row\. This addresses similar needs as those addressed by the traditionalOFFSETfunction, which offsets a computed number of rows and columns from a given cell coordinate\. Computed absolute indices are not supported because when users specify absolute indices, Kale must resolve them to row IDs at parse time\.

relativeRowRef⩴\\Coloneqq'\[' '\-' expr '\]'∣\\mid'\[' '\+' expr '\]'absoluteRowRef⩴\\Coloneqq'\[' INT '\]'relativeCellOrColRef⩴\\ColoneqqIDENT∣\\midIDENT '\[' \- expr '\]'∣\\midIDENT '\[' \+ expr '\]'absoluteCellOrColRef⩴\\ColoneqqIDENT '\[' INT '\]'Figure 3:Kale reference syntax\. Kale uses the ANTLR parser generator[Parr \(2024\)](https://arxiv.org/html/2608.26345#bib.bib16), which uses the first matching rule in cases of ambiguity\. Brackets, when present, always signify row indices or offsets\. Note how \+ and \- are used to specify that an index is relative rather than absolute\. Literal indices are required for absolute references; other expressions that evaluate to numbers have relative reference semantics\.Some spreadsheet systems \(including Excel and Google Sheets\) enable users to use a dedicated interface to give*names*to specific regions\. After doing so, formulas can refer to those ranges by name\. For example,SUM\(Sales\)refers to a region calledSalesthat was previously bound to a particular range\. To enable name\-based reference, Kale provides a*name bar*, a text field that enables users to name individual cells\. When the user wants to reference a cell in a formula, they can use either the location of the cell in the grid or the name of the cell\. Like Excel and Sheets, names move with cell contents on cut/paste operations\.*Unlike*Excel and Sheets, names also move during sort operations\. In Excel and Sheets, cell names are only visible when the cell is selected, and only in a dedicated name affordance\. In Kale, every named cell always shows its name so that users can easily use them when writing formulas\.[Figure4](https://arxiv.org/html/2608.26345#S2.F4)compares the name user interfaces in Excel and Kale\.

![Refer to caption](https://arxiv.org/html/2608.26345v1/names-Excel.png)\(a\)Named cells in Excel
![Refer to caption](https://arxiv.org/html/2608.26345v1/names-Kale.png)\(b\)Named cells in Kale

Figure 4:Excel only shows one name at a time\. Kale shows names in each named cell\.
### 2\.2Implementation

Kale is a web application implemented with Typescript and React\. Kale uses an off\-the\-shelf HTML/CSS/JavaScript table implementation, AG Grid[AG Grid Ltd\. \(2024\)](https://arxiv.org/html/2608.26345#bib.bib9)\. Kale uses AG Grid’s core spreadsheet features, which include columns, rows, cells, filtering, selection, and editing\. We also implemented some original features, including naming cells, representing and evaluating formulas, dependency graphing, and drag\-drop\.

## 3Preliminary Study of User Expectations

### 3\.1Methods

To understand users’ expectations regarding traditional spreadsheet behavior, we first conducted an IRB\-approved survey\-based preliminary study asking five participants to predict the results of several spreadsheet operations\. We recruited participants using posters and our own contacts\. All five participants were graduate students at a public R1 university, from fields of study including computer science, data science, and business analytics\. In each question, we provided participants with a scenario and several multiple\-choice options depicting possible results\. The first four questions asked about the behavior of traditional absolute references \(as implemented in Excel\)\. The remaining three questions asked about how formulas are updated in response to structural table changes\. Survey questions and results are displayed in[Table1](https://arxiv.org/html/2608.26345#S3.T1)\.

Table 1:Summary of survey questions and results from the preliminary study\.![Refer to caption](https://arxiv.org/html/2608.26345v1/Q5.png)Figure 5:Question five regarding traditional spreadsheet behavior\.
### 3\.2Results and Discussion

One participant answered all four questions about absolute references correctly; three got 3/4 correct; one got 2/4 correct\. No one correctly answered all of the questions about how references are updated when rows are inserted; two answered 2/3 correctly and the remaining three answered 1/3 correctly\.

Most participants had a correct understanding of traditional spreadsheet absolute references\. All but one participant responded to at least three of the four questions correctly\. However, for the questions about reference transformations on row insertion, there were only seven correct answers out of 15 responses, for a success rate of about 47%\. Although it is possible that more expert spreadsheet users might know the answers to these questions, spreadsheet systems are targeted at naive users, not only at experts; we view the semantics of references as fundamental to basic spreadsheet usage\. The latent relationship between references and referenced cells means that users who insert rows may be unaware of which formulas might reference those rows\. Combined, these two facts suggest that \(a\) when users write formulas, they write them without regard for how they will be updated when table structures change; \(b\) when users make structural changes to tables, they cannot conveniently assess whether their changes have broken any referencing formulas\.

## 4User Study

### 4\.1Methods

Given our observation in[section3](https://arxiv.org/html/2608.26345#S3)that users may be unaware of the effects of structural changes on formulas, we turned to assess the risk of bugs resulting from these misunderstandings and evaluate Kale’s efficacy as a solution\.

Specifically, we designed a user study aiming to explore the following research questions:

- •RQ1: To what extent do the risks described in[Section1](https://arxiv.org/html/2608.26345#S1)cause bugs in tasks using traditional spreadsheets?
- •RQ2: Is there a difference in the risk of encountering these bugs with Kale, as compared to traditional spreadsheets?
- •RQ3: Is there a difference in overall task correctness with Kale, as compared to traditional spreadsheets? Unlike RQ2, this considers errors beyond those caused by the risks that we have defined\.
- •RQ4: Is there a difference in task completion speed with Kale, as compared to traditional spreadsheets?

We conducted a between\-subjects, IRB\-approved study with 25 participants, who completed spreadsheet tasks using either Google Sheets or Kale\. Participants were recruited via postings on online student forums associated with two public R1 universities, as well as our own contacts and snowball sampling \(in which we asked participants to refer their contacts\)\. No participants from the preliminary study were included in this sample\. During recruitment, we obtained informed consent from participants and asked them to complete a survey asking about demographic information, including age, gender, occupation, educational background, field of study, and self\-estimated experience and frequency of usage with spreadsheets\. All user study sessions were conducted remotely via Zoom\. The study took an hour for each participant\. We compensated each participant with a $20 gift card\.

In order to obtain a balanced sample with approximately equal numbers of participants in both of our two conditions, we manually assigned participants to conditions based on their self\-estimated experience with spreadsheets and, as many of our participants were students, field of study\. We assigned 12 participants to use Sheets and 13 to use Kale\.

We provided participants with a document providing the details of each task and links to spreadsheets, one for each task, in Google Sheets or a web\-based implementation of Kale\. We selected Sheets for our traditional spreadsheet condition since it is available as a web application, allowing us to share task spreadsheets with participants via links without requiring them to download software\. Participants assigned to use Sheets were permitted to download the spreadsheet from Sheets and use any spreadsheet tool of their choice \(e\.g\. Excel or Numbers\); however, all 12 participants assigned to this condition elected to use Sheets\.

Participants assigned to use Kale were shown a tutorial video \(approx\. 6 minutes long\), and additionally had access to a Kale help document, which they could refer to for assistance with syntax and other tool\-specific functionalities\. Participants were allowed access to internet tools \(e\.g\. Google search\) during the study\. This was intended to enable participants assigned to Google Sheets to look for documentation and help online\. However, we disallowed participants from using generative AI tools \(e\.g\. ChatGPT\) to prevent them from generating formulas, which we felt could disrupt the formula\-writing process in which we are interested\.

While participants completed tasks, we recorded their screens and audio\. We also collected the final spreadsheets created by the participants\. This allowed us to analyze the end results of their tasks, focusing on occurrence of errors, and to note different formulas used by participants to complete the tasks\. There were no per\-task time limits\. However, the overall study duration was limited to an hour\.

### 4\.2Participants

We recruited 25 participants; 12 identified as female, 11 as male, one as non\-binary, and one declined to state\. Their average age was approximately 23, with a range from 18 to 27\. By their self\-described occupations, 11 participants were students\. The remaining participants were employed in diverse occupations, including project management, software engineering, music teaching, and cinema programming\. Participants also studied in or had graduated with degrees in a diverse set of fields, most common among them data science \(6\), cognitive science \(5\), and economics \(5\)\. While most participants majored in STEM fields, there were also participants who studied film, English literature, and music\.

Participants also reported their self\-estimated experience with spreadsheets, rated from one \(least experienced\) to ten \(most experienced\)\. The mean was approximately 5\.44, with a minimum of two and a maximum of nine\. This metric was used to assign participants to balanced groups; the mean self\-estimated experience was 5\.50 in the Sheets condition, out of 12 participants, and 5\.38 in the Kale condition, out of 13 participants\. Field of study was also used to inform balancing; participants were assigned such that, between conditions, there was a balance of participants from different fields, as well as broader areas like sciences, engineering, and humanities\.

### 4\.3Tasks

We designed four tasks, T1, T2, T3, and T4, to evaluate the risk of users introducing bugs in their spreadsheet via R1, R2, R3, R4, and R5\. To improve their real\-world plausibility, spreadsheets modified during these tasks were based off of real spreadsheets from the EUSES spreadsheet corpus[Fisher and Rothermel \(2005\)](https://arxiv.org/html/2608.26345#bib.bib8)\. Each task was also coupled with a real\-world scenario involving modifying the spreadsheet and sharing it with a fictional coworker or colleague; this scenario was intended to encourage participants to ensure their formulas were correct, even after structural transformations, without explicitly requiring them to double\-check the formulas\. Tasks were divided into two to six parts, which participants were instructed to complete in order\.

Participants in both conditions were given the same tasks\. Some minor changes were required to adapt the spreadsheet tasks to Kale; most notably, T1, T2, and T3 asked spreadsheet users to fill in a table of summary statistics, while Kale users were instead asked to fill in the last row of the sheet, due to Kale’s requirement that all sheets be structured as a single table\. Since Kale disallows spaces in column names, some columns were given minor naming changes\. One part of T3 also asked participants to bold a row; this instruction was omitted from the Kale version of this task, as bolding text is not implemented in Kale\.

We designed each task to enable evaluation of specific reference transformation risks as follows:

![Refer to caption](https://arxiv.org/html/2608.26345v1/t1sheets.png)
![Refer to caption](https://arxiv.org/html/2608.26345v1/t1kale.png)

Figure 6:The Classes task \(T1\), shown in Sheets on the top and Kale on the bottom\.T1: Classes \(risk R1\): We provided participants with a spreadsheet of classes for a professional development company, with each entry in the table consisting of three rows\.111The spreadsheet used in T1 is based closely on the spreadsheet02rise\.xlsfrom thefinancialsection of the EUSES corpus\. Our version is significantly scaled down, with many rows and columns removed\.[Figure6](https://arxiv.org/html/2608.26345#S4.F6)shows this task in Google Sheets and Kale\. In Part 1 of this task, we asked participants to use formulas to compute summary statistics across columns of the table, including the total of one column \(“NUMBER OF CLASS DAYS”\), and the averages of two columns \(“PUBLIC COST” and “COMPANY COST”\)\. In Part 2, we then asked participants to add new data to the table, consisting of three new entries, from another sheet, and to insert the entries into the table such that it remained sorted chronologically\. Since the correct placement of two of these entries involves adding new rows to the beginning and end of the table, this transformation causes formulas in traditional spreadsheets referencing only the existing range of rows to break, incurring risk R1\.

T2: Gradebook \(risks R1, R2, and R4\): We tasked participants with modifying a mock teacher’s gradebook with test and project scores for students in different classes\.222The spreadsheet in T2 is based on a sheet in7\_gradebook\_xls\.xlsxfrom thegradessection of the EUSES corpus\. Compared to the original, we removed some rows and columns, and filled the table with mock data\.[Figure7](https://arxiv.org/html/2608.26345#S4.F7)displays this task in Google Sheets\. This was the only task to involve multiple risks; across the formulas and structural transformations involved in this task, we identified four potential risks: two instances of R1, and one of R2 and R4\. In Part 1, we asked participants to add a new “Intro Project” column, located in another sheet, into the table\. The original \(unmodified\) spreadsheet includes a column titled “Project Average” taking the average across columns in the Projects category\. This formula is written in Sheets using a range, and in Kale using an average of individual cells\. In both conditions, the new column “Intro Project” added in Part 1 is not automatically included in the formula, incurring risk R1\. In Part 2, participants wrote a formula to compute the average test score across all students, as well as a formula in Part 3 to compute the average test score of only Class A\. In Part 4, participants added new rows corresponding to students from a new class, Class E, to the table\. The rows added to the table during this part may not be included in the average of all student test scores from Part 1 if the formula is written using a range across only the existing rows\. This incurs risk R1 for a second time\. In Part 5, participants then computed the average of three named cells \(named ranges in Sheets\) holding scores of “fan favorite” projects\. Finally, in Part 6 participants sorted the table in alphabetical order by student last name, permuting the rows\. If the formula from Part 3 computing the average test score of students in Class A was written using a reference to the rectangular range of cells containing the scores of students in Class A, sorting caused this formula to reference different data, incurring risk R2\. In addition, if named ranges were used in Sheets to compute the average score of fan favorite projects in Part 5, sorting also produced another risk; in Sheets, named ranges do not move when rows are sorted, leading to risk R4\.

![Refer to caption](https://arxiv.org/html/2608.26345v1/t2sheetsbig.png)

Figure 7:The Gradebook task \(T2\), shown in Sheets\.T3: Payroll \(risk R3\): We provided participants with a table of pay rates at an IT company for different roles, across four time periods\.333The spreadsheet in T3 is nearly identical to a sheet inIT\_Schedule\_Rates\.xlsfrom thedatabasesection of the EUSES corpus\. Compared to the original, we removed two hidden columns\.[Figure1](https://arxiv.org/html/2608.26345#S1.F1)shows this task in Kale\. In Part 1, we asked participants to write a formula computing the average pay across four different “software engineering” roles in the most recent time period \(the rightmost column\)\. In Part 2, we asked participants to swap the values in two rows, one of which corresponded to the bottommost software engineering role\. Finally, in Part 3 we asked participants to move the row for the topmost software engineering role up to the first row of the table, and, in the Sheets condition, bold the row\. If the average pay across software engineering roles is computed using a range of cells, this task involves moving the first and last row in that range\. If these transformations are performed in Sheets via drag/drop or cut/paste, the range expands to incorrectly include the cells between, incurring risk R3\.

T4: Aquarium \(risk R5\): We asked participants to fill in an incomplete spreadsheet predicting the populations of fish in different tanks at an aquarium over a period of five months, with one column for each month444The spreadsheet in T4 is loosely inspired byCANTILEV\.xlsfrom thefilbysection of the EUSES corpus\. The use case for and data from this spreadsheet are entirely different from ours; however, we attempted to replicate the original spreadsheet’s structure and layout \(e\.g\. its practice of storing constants in a separate table\)\.\. In Part 1, participants first used formulas to fill two columns with the values in the previous \(left\-adjacent\) column, multiplied by a growth rate and rounded to the nearest integer\. We provided participants with a named cell in the sheet holding the value of the growth rate, as well as a formula to reuse and adapt:=ROUND\(n \* growth\_rate, 0\)\. In Part 2, we asked participants to fill in the remaining two columns according to the same pattern, but with an added flat value changing between the two months\. The value to add each month was stored in a separate table in the Sheets condition, and in additional columns in the Kale condition\. Unlike previous tasks, T4 was designed to assess risks associated with confusion between relative and absolute references; if the flat value to add each month is accessed using a relative reference, then drag\-filling or copy\-pasting the formula will not correctly fill in the rest of the column, leading to risk R5\.

![Refer to caption](https://arxiv.org/html/2608.26345v1/t4sheets.png)Figure 8:The Aquarium task \(T4\), shown in Sheets\.
### 4\.4Results

To answer our research questions, we collected participants’ final spreadsheets\. Three participants “timed out” and did not complete all tasks before the end of the study duration\. The resulting incomplete or unattempted tasks are excluded from all analyses, as well as one task where a participant using Sheets was, due to an error, sent a link to a spreadsheet for task T4 that had already been completed\. One participant using Sheets timed out after completing three tasks, while two participants using Kale both timed out after completing two tasks\.

To assess task correctness, we manually compared participants’ spreadsheets to a reference solution\. We measured correctness separately for each part of a given task\. Since we are interested primarily in formula errors, we limit our analysis to parts of tasks involving writing or editing formulas\. Across all four tasks, there were eight such formula\-editing parts\. For incorrect formulas, we recorded whether they were caused by a risk or some other error\. For incorrect formulas exhibiting both a risk and another error, we considered them incorrect due to the risk\.[Figure9](https://arxiv.org/html/2608.26345#S4.F9)displays the percentage of participants in each part and condition that wrote correct formulas, wrote incorrect formulas due to risks, and wrote incorrect formulas due to a different error\. Kale exhibited a higher correctness rate in five task parts, as well as a lower or equal rate of non\-risk\-related errors in another five parts\. We conducted a Wilcoxon exact test measuring the effect of condition on the total number of correct formula tasks, finding that the difference in total correctness is not statistically significant \(p=\.025p=\.025\)555As described later, we conduct seven other hypothesis tests on risk\-related error rate\. Since risk\-related error rate is not independent of task correctness, we perform a Bonferroni correction on these eight hypothesis tests to control the probability of a type I error\. Our new significance level for these hypothesis tests only isα=0\.05/8=0\.00625\\alpha=0\.05/8=0\.00625\.\.

![Refer to caption](https://arxiv.org/html/2608.26345v1/formula_correctness_nuanced.png)Figure 9:Breakdown of participant correctness percentages for formula\-related task parts, by condition and task part\.To assess task completion times, we recorded start and end times for each task\.[Figure10](https://arxiv.org/html/2608.26345#S4.F10)displays the distributions of task completion times in each condition\. According to a Wilcoxon exact test, Kale users experienced a statistically significant speedup in Task 3 \(p=\.049p=\.049\), but differences in timing were not significant in the other tasks \(p=\.69p=\.69,p=\.85p=\.85, andp=\.15p=\.15for Tasks 1, 2, and 4 respectively\)\.

![Refer to caption](https://arxiv.org/html/2608.26345v1/task_timing.png)Figure 10:Distributions of task completion time in minutes by task and condition\.Lower is better\.To assess the occurrence of risks, we manually inspected participants’ spreadsheets and assigned occurrences of each risk to one of five categories, abbreviated A, C, I, N, and E:

- •Risk avoided \(A\): The participant avoided this risk by writing a formula that prevented the bug from occurring\.
- •Risk fixed correctly \(C\): The participant encountered the associated bug, but noticed and correctly fixed it\.
- •Risk fixed incorrectly \(I\): The participant encountered the associated bug and noticed it, editing the formula, but did not correctly fix the bug\.
- •Risk not fixed \(N\): The participant encountered the associated bug and did not notice it, or otherwise did nothing to edit the formula\.
- •Risk not encountered due to error \(E\): The participant inserted an error elsewhere during the task that prevented them from encountering the risk\.

The counts of participants in each category for each task\-risk combination is shown in[Table2](https://arxiv.org/html/2608.26345#S4.T2)\. For participants who encountered risks, we also noted the ways in which they caused them; for those who avoided risks, ways in which they wrote formulas to avoid them; and for participants who incorrectly fixed risks, ways in which the fixes were incorrect\.

We additionally computed the rate at which participants in each condition ultimately produced errors related to each risk\. We grouped categories I and N together to produce a count of participants who submitted formulas with errors due to the corresponding risks, and grouped A and C together to count participants who did not produce risk\-related errors\. We conducted hypothesis tests using these counts \(comparingI\+NI\+NagainstA\+CA\+C\)\. We additionally calculated the rate of risk\-related errors for each task\-risk combination asI\+NA\+C\+I\+N\\frac\{I\+N\}\{A\+C\+I\+N\}\. That is, we divided the number of participants who encountered the risk and did not correctly fix it by the total number of participants, excludingEE, the group of participants who could not have encountered the risk due to other errors\.[Figure11](https://arxiv.org/html/2608.26345#S4.F11)shows this risk\-related error rate for each task\-risk combination in both conditions\. We note several observations, which we organize by risk\.

Table 2:Summary of risk categories in both conditions\. A: avoid; C: fixed correctly; I: fixed incorrectly; N: not fixed; E: not encountered due to errorR1: Insertions adjacent to ranges are ambiguous\. In Sheets, all three instances of risk R1 were encountered by some participants\. These participants’ responses to this risk varied; in all three instances, some participants were able to identify and fix the error, but some others did not notice or otherwise attempt a fix\. In T1, four participants attempted to fix the error but only edited*one*end of a range that had extended in both directions\. One participant in T1 and two in T2 were able to avoid instances of this risk using Sheets by referencing whole columns instead of ranges of cells \(e\.g\. referencing the whole column A with rangeA:A\)\. In Kale, all 13 participants were able to entirely avoid the first and third instances of risk R1 by writing formulas referencing whole columns\. However, the second instance of this risk, a formula averaging only part of each row, went undetected by most participants, and, unlike the other cases of risk R1, was not automatically fixed by whole\-column references in either condition\. Kale fully eliminated errors caused by the first and third instances of R1, but in the second instance, the rate of risk\-related errors was slightly higher with Kale\. Aχ2\\chi^\{2\}test shows that Kale yielded a statistically significant decrease in the rate of risk\-related errors in the first and third instances \(χ2​\(1,N=25\)=6\.67,p=\.0003\\chi^\{2\}\(1,N=25\)=6\.67,p=\.0003andχ2​\(1,N=23\)=10\.22,p=\.0014\\chi^\{2\}\(1,N=23\)=10\.22,p=\.0014, respectively\), but finds no significant relationship in the second \(χ2​\(1,N=25\)=0\.01,p=\.93\\chi^\{2\}\(1,N=25\)=0\.01,p=\.93\)\.

R2: Sorting permutes referenced data\. In Sheets, risk R2 caused five participants to submit errors, compared to four who were able to fix the error\. Three participants did not encounter the risk, due to sorting the table incorrectly\. No Sheets participants wrote formulas avoiding this risk\. However, with the exception of one participant who skipped the sorting step, all 12 other Kale participants were able to avoid the risk using Kale’s adjustment of references to sorted data\. Aχ2\\chi^\{2\}test shows that Kale yielded a statistically significant decrease in the rate of errors related to this risk \(χ2​\(1,N=21\)=10\.69,p=\.0011\\chi^\{2\}\(1,N=21\)=10\.69,p=\.0011\)\.

R3: Moving cells changes references\. In Sheets, seven participants were able to successfully identify and fix risk R3\. Eight Kale users were able to fully avoid the risk using Kale’s automatic adjustment of references in moved row\. However, for two participants, this behavior introduced a new, related risk\. These participants swapped the numerical row values in Part 2 by exchanging the job titles and moving the rows, rather than by copy\-pasting the numerical values; Kale automatically updated references to these rows during the move, causing the referenced values to remain the same instead of being replaced\. In categorizing this error, we decided to consider it an instance of R3, as it was generated by the same circumstances as R3 would be in Sheets\. Despite this threat, aχ2\\chi^\{2\}test finds no significant relationship between condition and rate of errors related to this risk \(χ2​\(1,N=20\)=0\.27,p=\.60\\chi^\{2\}\(1,N=20\)=0\.27,p=\.60\)\.

R4: Names do not always move alongside data\. In Sheets, the majority of participants \(7/12\) encountered R4, and once encountered no participant was able to identify and fix the error\. Two participants were able to avoid the error by using the literal values of the named ranges in their formula, rather than referencing the named ranges themselves\. In Kale, all 13 participants were able to avoid the error, since Kale automatically moves named cells during sorting\. Aχ2\\chi^\{2\}test shows that Kale yielded a statistically significant decrease in the rate of errors related to this risk \(χ2​\(1,N=22\)=17\.99,p<\.0001\\chi^\{2\}\(1,N=22\)=17\.99,p<\.0001\)\.

R5: References are relative by default\. In T4, across both conditions, all 22 participants attempting the task were able to avoid risk R5\. Participants avoided this risk using a variety of methods, including hard\-coding the added value instead of referencing a cell, using absolute references, or by defining new named cells\. Since all participants had the same outcome, we do not perform aχ2\\chi^\{2\}test for this risk\.

![Refer to caption](https://arxiv.org/html/2608.26345v1/risk_related_error.png)Figure 11:Rates of risk\-related error by condition for each task\-risk combination\.Lower is better\.
### 4\.5Discussion

To answer RQ1 regarding the impact of our five risks, we refer to the rates of risk\-related errors\. To put these error rates into perspective, Panko[Panko \(2015\)](https://arxiv.org/html/2608.26345#bib.bib4)hypothesized that the average rate at which users make errors in spreadsheet cells is within the range of 1\-5%, with prior studies discovering an average error rate of 3\.9%\. With the exception of R5, the rates at which participants using Sheets in our study inserted errors due to these risks \(50\-83% for R1, 56% for R2, 30% for R3, and 78% for R4\) were orders of magnitude larger than Panko’s estimated global cell error rate, suggesting that risks R1\-R4 all present a distinct danger to spreadsheet users\. In contrast, no participants encountered R5 \(references are relative by default\), suggesting either that this risk is not a danger to users, or that our task was ineffectively designed to assess the risk\.

To address RQ2, we compare the incidence of risk\-related errors between participants using Sheets and Kale\. We found that, in four out of seven instances of risks, Kale completely eliminated risk\-related errors that were present with rates of over 50% in Sheets\. Kale significantly reduces the occurrence of three out of five risks: R1, R2, and R4\. Simultaneously, results for RQ3 and RQ4 show that participants do not significantly sacrifice correctness or speed by using Kale\.

However, our results also reveal opportunities for tool improvement and further investigation\. We found that Kale was not able to eliminate one variant of errors caused by R1: specifically, whole\-column and whole\-row references do not allow a range to automatically adjust to new data if that range does not span a whole table row or column\. We also observed that, in a specific situation encountered by two participants in T3, Kale’s sorting behavior actually introduced new errors\. This may be because users’ expectations have been shaped by their experience with traditional spreadsheets, which have subtly different sorting behavior\. These issues partially explain Kale’s failure to significantly prevent R3 in T3 and the first instance of R1 in T2, as reflected in both our hypothesis tests and the error rates shown in[Figure11](https://arxiv.org/html/2608.26345#S4.F11)\.

## 5Corpus Study

### 5\.1Methods

Because Kale does not support arbitrary rectangular range references, it is possible that some spreadsheet tasks are difficult with Kale\. If a formula needs to refer to only part of a row or column, then it might suffice to write the formula using multiple references\. Otherwise, it might be necessary to restructure\.

To assess to what extent users might need to make major changes to their documents if they had used Kale to create them rather than a traditional spreadsheet, we conducted a corpus study\. We began with the EUSES corpus[Fisher and Rothermel \(2005\)](https://arxiv.org/html/2608.26345#bib.bib8), which includes 4,233 Excel spreadsheets\. After removing duplicates and those with minimal differences, we refined our dataset to 1,726 unique spreadsheets\. We categorized these into three groups based on their formulas:

- •Spreadsheets that included range references \(e\.g\.,SUM\(A1:B10\)\): 677 spreadsheets \(39%\)\.
- •Spreadsheets with cross\-sheet references \(e\.g\.,Sheet1\!C3\): 50 spreadsheets \(3%\)\.
- •Other spreadsheets \(references were limited to single cells\): 999 spreadsheets \(58%\)\.

Because of our interest in evaluating Kale, we sampled 50 spreadsheets from the first group, containing range references\. We also sampled 10 spreadsheets with formulas but only single\-cell references for comparison purposes\. Then, three co\-authors divided the documents among themselves and manually converted the documents to Kale, recording the time and changes required\.

### 5\.2Results and Discussion

We were able to convert all 60 spreadsheets to use Kale\. The time required ranged from 4 to 44 minutes, with a mean of 18 minutes and a median of 15 minutes\. Spreadsheets with few range references were trivially easy to convert to Kale, while spreadsheets with many range references were sometimes challenging\. Due to Kale’s limited function library, we did not attempt to translate calls to unsupported functions\.

This evaluation is limited because it did not include documents that use cross\-table references\. However, since those only comprise 3% of the corpus, and the mean time was 18 minutes, we conclude that reference\-related restrictions in Kale are unlikely to present significant barriers to real\-world success\.

## 6Related Work

The flexibility and ease\-of\-use of spreadsheets have contributed to their global commercial success, but have also made them prone to errors[Chalhoub and Sarkar \(2022\)](https://arxiv.org/html/2608.26345#bib.bib11)\. From a total of 85 intensive inspection studies, it was found that 94% of the spreadsheets contained errors[Panko \(2015\)](https://arxiv.org/html/2608.26345#bib.bib4)\. In general, past studies have audited existing spreadsheets; our study observes participants using spreadsheets to see whether they identify errors as they occur\.

Understanding the root causes of spreadsheet errors is crucial for addressing them\. Numerous studies have explored common errors and their causes, considering both limitations of human cognitive processes[Panko \(2013\)](https://arxiv.org/html/2608.26345#bib.bib17)and specific kinds of errors\. Panko[Panko \(2009\)](https://arxiv.org/html/2608.26345#bib.bib7)categorized spreadsheet errors into two types: blameless errors from innocent mistakes and culpable violations of laws or corporate practices\. The causes of these errors are due to misinterpretation of domain knowledge, incorrect algorithmic expression, and implementation mistakes, which include slips and lapses during execution\.

In an effort to develop means of preventing and detecting errors, several other researchers have categorized spreadsheet errors\. In a field study, Caulkins et al\.[Caulkins et al\. \(2006\)](https://arxiv.org/html/2608.26345#bib.bib12)found that executives and managers frequently encounter inaccurate data, errors inherited from reuse of spreadsheets, model errors \(including structural errors and errors of omission\), and errors in the use of functions\. Similarly, Badame et al\.[Badame and Dig \(2012\)](https://arxiv.org/html/2608.26345#bib.bib13)explored spreadsheet refactoring and discovered that formulas often suffer from the same issues seen in professional software: hardcoded constants, duplicated expressions, unnecessary complexity, and unsanitized input\. Barowy et al\.[Barowy et al\. \(2018b\)](https://arxiv.org/html/2608.26345#bib.bib3)attributed some errors to the ease of use of drag\-fill, which can result in correct references\. Lastly, Reinhardt et al\.[Reinhardt and Pillay \(2004\)](https://arxiv.org/html/2608.26345#bib.bib10)conducted a case study on computer literacy students, and found that their errors can be classified into either conceptual\-related errors, or mathematical and logical related errors\. None of these categories distinguish errors inserted via structural changes in spreadsheets from other kinds of errors\. In this paper, we leverage this distinction to identify a new risk and possible solution space\.

Several tools have also been developed to identify and quantify errors\. Schalkwijk et al\.[Schalkwijk et al\. \(2016\)](https://arxiv.org/html/2608.26345#bib.bib18)proposed PerfectXL, a tool that visualizes spreadsheet dependencies and determines possible errors in spreadsheets based on previous findings\. Barowy et al\.[Barowy et al\. \(2018a\)](https://arxiv.org/html/2608.26345#bib.bib15)developed a static analysis tool, ExceLint, which uses the rectangular layout of spreadsheets to identify formula errors\. Badame et al\.[Badame and Dig \(2012\)](https://arxiv.org/html/2608.26345#bib.bib13)\. implemented REFBOOK, a plugin for Excel that implements seven refactorings that reliably remove spreadsheet smells, patterns or characteristics in spreadsheets that might indicate deeper problems\. Rather than rectifying errors that appear in Excel, Kale provides users with a new platform for spreadsheet creation, manipulation, and error detection\. Many related approaches to automated spreadsheet QA are discussed in Jannach et al\.’s[Jannach et al\. \(2014\)](https://arxiv.org/html/2608.26345#bib.bib19)survey, including tools for formula visualization, static analysis, fault localization, testing, model\-driven development, and design and maintenance of spreadsheets\. Most similar to Kale are tools that provide enhanced safety guarantees in spreadsheets as they are being created, rather than testing, debugging, or analysis of existing spreadsheets\.

Some approaches use features in existing spreadsheet tools; for example, Panko et al\.[Panko \(2008\)](https://arxiv.org/html/2608.26345#bib.bib14)proposed cell protection, or restricting modifications to pre\-specified input cells, and re\-keying data, a traditional data verification approach that reduces data input errors by requiring data to be entered twice in two separate input sections, allowing error detection by highlighting differences between two blocks of input data\.

Chalhoub et al\.[Chalhoub and Sarkar \(2022\)](https://arxiv.org/html/2608.26345#bib.bib11)discovered that without formal types or data structures, spreadsheets suffer from classes of errors that in traditional programming languages are easily detected and prevented, and propose to introduce data structures that aim to strike a balance between the freedom and flexibility of the traditional grid, and the safety and power of formal types and structures\. However, introducing these restrictions could restrict users’ ability to flexibly use the cell grid\. This insight is particularly relevant to Kale, as one potential drawback is that it limits users to structuring their spreadsheets as separate tables\.

A number of tools have also, like Kale, been designed to prevent errors by enforcing structure on spreadsheet design\. Many of these existing tools involve model\-driven development: the use of a separate, abstract representation of a spreadsheet to ensure certain properties about its function, especially by generating provably correct formulas\. Early work by Isakowitz et al\.[Isakowitz et al\. \(1995\)](https://arxiv.org/html/2608.26345#bib.bib27)proposed the separation of spreadsheets into logical and physical layers representing the spreadsheet’s functionality and implementation\. Paine[Paine \(2008\)](https://arxiv.org/html/2608.26345#bib.bib26)introduced Model Master, a tool capable of compiling spreadsheets to and from a textual, declarative programming language\. Erwig et al\.[Erwig et al\. \(2005\)](https://arxiv.org/html/2608.26345#bib.bib20);[Erwig et al\. \(2006\)](https://arxiv.org/html/2608.26345#bib.bib21)introduced Gencel, a system for developing visual spreadsheet templates which can be used to automatically generate spreadsheets with defined update operations\. Similarly to Kale, Gencel ensures that cell references stay valid under structural transformations\. Engels and Erwig[Engels and Erwig \(2005\)](https://arxiv.org/html/2608.26345#bib.bib22)also introduced ClassSheets, a system for visually developing object\-oriented models of spreadsheets\. In a series of related work, Cunha et al\.[Cunha et al\. \(2011b\)](https://arxiv.org/html/2608.26345#bib.bib23);[Cunha et al\. \(2011a\)](https://arxiv.org/html/2608.26345#bib.bib24);[Cunha et al\. \(2012\)](https://arxiv.org/html/2608.26345#bib.bib25)developed systems for bidirectionally syncing a ClassSheet model with a generated spreadsheet\. While all of the spreadsheet modeling tools we have discussed aim to solve problems similar to those addressed by Kale, the tools are fundamentally different from the user’s perspective\. Model\-driven development requires the user to interact with a separate representation of their spreadsheet, while Kale users interact with a single, unified representation\. Model\-driven development tools generate traditional spreadsheets with correctness properties–Kale aims to instead achieve these properties by introducing constraints on the structure of the spreadsheets and formulas with which users directly interact\. Our work represents a first step into this design space; future work will be required to understand the usability difference between these paradigms\.

## 7Limitations and Future Work

Further study is needed to assess the real\-world risks that structural changes pose in the context of spreadsheetauthoringtasks, beyond modification of existing spreadsheets\. A more open\-ended study could elucidate whether users tend to build spreadsheets that are fragile in the way ours were\. The tasks in this study were designed to produce the associated risks; future work is required to measure how often these situations appear during real\-world tasks\. Additionally, while our corpus study demonstrates that replicating real spreadsheets ispossiblein Kale, we would also like to evaluate how Kale’s structural constraints impact real spreadsheet authors and viewers\. The effect on users of splitting spreadsheets into separate, explicit tables is unclear, and warrants further investigation\.

Kale is a prototype system\. One key feature we did not evaluate in our study \(due to an immature implementation\) is*queries*, which would enable formulas to reference a subset of a range\. For example,AVERAGE\(Salary\[Experience \> 10\]\)might compute the average salary of employees with over 10 years of experience\. Another feature that we did not evaluate is cross\-table references, which in practice may be important for representing spreadsheets with more diverse structures\.

## 8Conclusion

In this paper, we showed evidence that structural changes in spreadsheets may present a common and serious risk to spreadsheet users\. Kale represents an approach to eliminate these risks posed by traditional spreadsheet reference representation and semantics\. Kale restricts the forms references can take and re\-defines the*absolute*and*relative*distinction, allowing users to specify when writing formulas how they should be updated for structural changes\. Evidence from our study suggests that adopting safer spreadsheet reference behavior may significantly reduce the incidence of spreadsheet errors\.

## 9Declarations

### 9\.1Funding

Not applicable\.

### 9\.2Ethical Approval

All studies were approved by our institution’s Institutional Review Board \(IRB\)\.

### 9\.3Informed Consent

All data collected from study participants was obtained with informed consent\.

### 9\.4Author Contributions

Michael Coblenz and Jacob Yim contributed to system design and implementation, user study design, and paper writing\. Ajinkya Bokade, Mounika Padala, Julia Epshtein, Priyanka Bhatia, Piyush Chauhan, Simran Gill, and Grishma Gurbani contributed to system design and implementation\. Aniket Gupta, Vaibhav Khetan, Arushi Munjal, and Jeffery Tung contributed to the corpus study\. Joanna Yang, Mounika Padala, and Priyanka Bhatia contributed to the user study design\.

### 9\.5Data Availability Statement

A replication package containing all data supporting the findings of our user study, as well as tasks and materials provided to participants, is available at the following URL:[https://github\.com/ucsd\-salad/kale\-replication](https://github.com/ucsd-salad/kale-replication)\. The Kale system is available as an open\-source project at the following URL:[https://github\.com/ucsd\-salad/Kale](https://github.com/ucsd-salad/Kale)\.

### 9\.6Conflict of Interest

Not applicable\.

## References

- AG Grid Ltd\. \(2024\)AG Grid Ltd\.AG grid\(Website\)External Links:[Link](https://ag-grid.com/)Cited by:[§2\.2](https://arxiv.org/html/2608.26345#S2.SS2.p1.1)\.
- Badame and Dig \(2012\)S\. Badame and D\. DigRefactoring meets spreadsheet formulas\.In2012 28th IEEE International Conference on Software Maintenance \(ICSM\),Vol\.,pp\. 399–409\.External Links:[Document](https://dx.doi.org/10.1109/ICSM.2012.6405299),ISSN 1063\-6773Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p3.1),[§6](https://arxiv.org/html/2608.26345#S6.p4.1)\.
- Barowyet al\.\(2018a\)D\. W\. Barowy, E\. D\. Berger, and B\. ZornExceLint: automatically finding spreadsheet formula errors\.Proc\. ACM Program\. Lang\.2\(OOPSLA\)\.External Links:[Link](https://doi.org/10.1145/3276518),[Document](https://dx.doi.org/10.1145/3276518)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p4.1)\.
- Barowyet al\.\(2018b\)D\. W\. Barowy, E\. D\. Berger, and B\. ZornExceLint: automatically finding spreadsheet formula errors\.Proc\. ACM Program\. Lang\.2\(OOPSLA\)\.External Links:[Link](https://doi.org/10.1145/3276518),[Document](https://dx.doi.org/10.1145/3276518)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p3.1)\.
- Caulkinset al\.\(2006\)J\. Caulkins, E\. Morrison, and T\. WeidemannSpreadsheet errors and decision making: evidence from field interviews\.Journal of End User Computing19,pp\. 1–23\.Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p3.1)\.
- Chalhoub and Sarkar \(2022\)G\. Chalhoub and A\. Sarkar“It’s freedom to put things where my mind wants”: understanding and improving the user experience of structuring data in spreadsheets\.InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems,CHI ’22,New York, NY, USA\.External Links:ISBN 9781450391573,[Link](https://doi.org/10.1145/3491102.3501833),[Document](https://dx.doi.org/10.1145/3491102.3501833)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p1.1),[§6](https://arxiv.org/html/2608.26345#S6.p6.1)\.
- Cunhaet al\.\(2012\)J\. Cunha, J\. P\. Fernandes, J\. Mendes, H\. Pacheco, and J\. SaraivaBidirectional Transformation of Model\-Driven Spreadsheets\.InTheory and Practice of Model Transformations,D\. Hutchison, T\. Kanade, J\. Kittler, J\. M\. Kleinberg, F\. Mattern, J\. C\. Mitchell, M\. Naor, O\. Nierstrasz, C\. Pandu Rangan, B\. Steffen, M\. Sudan, D\. Terzopoulos, D\. Tygar, M\. Y\. Vardi, G\. Weikum, Z\. Hu, and J\. De Lara \(Eds\.\),Vol\.7307,pp\. 105–120\(en\)\.Note:Series Title: Lecture Notes in Computer ScienceExternal Links:ISBN 978\-3\-642\-30475\-0 978\-3\-642\-30476\-7,[Link](http://link.springer.com/10.1007/978-3-642-30476-7/_7),[Document](https://dx.doi.org/10.1007/978-3-642-30476-7%5F7)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p7.1)\.
- Cunhaet al\.\(2011a\)J\. Cunha, J\. Mendes, J\. Saraiva, and J\. P\. FernandesEmbedding and evolution of spreadsheet models in spreadsheet systems\.In2011 IEEE Symposium on Visual Languages and Human\-Centric Computing \(VL/HCC\),pp\. 179–186\.Note:ISSN: 1943\-6106External Links:[Link](https://ieeexplore.ieee.org/document/6070396/),[Document](https://dx.doi.org/10.1109/VLHCC.2011.6070396)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p7.1)\.
- Cunhaet al\.\(2011b\)J\. Cunha, J\. Visser, T\. Alves, and J\. SaraivaType\-Safe Evolution of Spreadsheets\.InFundamental Approaches to Software Engineering,D\. Giannakopoulou and F\. Orejas \(Eds\.\),Vol\.6603,pp\. 186–201\(en\)\.Note:Series Title: Lecture Notes in Computer ScienceExternal Links:ISBN 978\-3\-642\-19810\-6 978\-3\-642\-19811\-3,[Link](http://link.springer.com/10.1007/978-3-642-19811-3/_14),[Document](https://dx.doi.org/10.1007/978-3-642-19811-3%5F14)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p7.1)\.
- Engels and Erwig \(2005\)G\. Engels and M\. ErwigClassSheets: automatic generation of spreadsheet applications from object\-oriented specifications\.InProceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering,Long Beach CA USA,pp\. 124–133\(en\)\.External Links:ISBN 978\-1\-58113\-993\-8,[Link](https://dl.acm.org/doi/10.1145/1101908.1101929),[Document](https://dx.doi.org/10.1145/1101908.1101929)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p7.1)\.
- Erwiget al\.\(2005\)M\. Erwig, R\. Abraham, I\. Cooperstein, and S\. KollmansbergerAutomatic generation and maintenance of correct spreadsheets\.InProceedings of the 27th international conference on Software engineering,ICSE ’05,New York, NY, USA,pp\. 136–145\.External Links:ISBN 978\-1\-58113\-963\-1,[Link](https://dl.acm.org/doi/10.1145/1062455.1062494),[Document](https://dx.doi.org/10.1145/1062455.1062494)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p7.1)\.
- Erwiget al\.\(2006\)M\. Erwig, R\. Abraham, S\. Kollmansberger, and I\. CoopersteinGencel: a program generator for correct spreadsheets\.Journal of Functional Programming16\(03\),pp\. 293\(en\)\.External Links:ISSN 0956\-7968, 1469\-7653,[Link](http://www.journals.cambridge.org/abstract/_S0956796805005794),[Document](https://dx.doi.org/10.1017/S0956796805005794)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p7.1)\.
- Fisher and Rothermel \(2005\)M\. Fisher and G\. RothermelThe euses spreadsheet corpus: a shared resource for supporting experimentation with spreadsheet dependability mechanisms\.InProceedings of the First Workshop on End\-User Software Engineering,WEUSE I,New York, NY, USA,pp\. 1–5\.External Links:ISBN 1595931317,[Link](https://doi.org/10.1145/1083231.1083242),[Document](https://dx.doi.org/10.1145/1083231.1083242)Cited by:[§1](https://arxiv.org/html/2608.26345#S1.p14.1),[§4\.3](https://arxiv.org/html/2608.26345#S4.SS3.p1.1),[§5\.1](https://arxiv.org/html/2608.26345#S5.SS1.p2.1)\.
- Gislason \(2018\)H\. GislasonExcel vs\. google sheets usage — nature and numbers\(Website\)External Links:[Link](https://medium.grid.is/excel-vs-google-sheets-usage-nature-and-numbers-9dfa5d1cadbd)Cited by:[§1](https://arxiv.org/html/2608.26345#S1.p1.1)\.
- Isakowitzet al\.\(1995\)T\. Isakowitz, S\. Schocken, and H\. C\. LucasToward a logical/physical theory of spreadsheet modeling\.ACM Transactions on Information Systems13\(1\),pp\. 1–37\(en\)\.External Links:ISSN 1046\-8188, 1558\-2868,[Link](https://dl.acm.org/doi/10.1145/195705.195708),[Document](https://dx.doi.org/10.1145/195705.195708)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p7.1)\.
- Jannachet al\.\(2014\)D\. Jannach, T\. Schmitz, B\. Hofer, and F\. WotawaAvoiding, finding and fixing spreadsheet errors – a survey of automated approaches for spreadsheet qa\.Journal of Systems and Software94,pp\. 129–150\.External Links:ISSN 0164\-1212,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.jss.2014.03.058),[Link](https://www.sciencedirect.com/science/article/pii/S0164121214000788)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p4.1)\.
- JPMorgan Chase & Co\. \(2013\)JPMorgan Chase & Co\.Report of JPMorgan Chase & Co\. management task force regarding 2012 cio losses\(Website\)External Links:[Link](https://ypfs.som.yale.edu/node/2821)Cited by:[§1](https://arxiv.org/html/2608.26345#S1.p1.1)\.
- Paine \(2008\)J\. PaineEnsuring Spreadsheet Integrity with Model Master\.arXiv\.Note:arXiv:0801\.3690 \[cs\]Comment: 15 pages; substantive references; code examplesExternal Links:[Link](http://arxiv.org/abs/0801.3690),[Document](https://dx.doi.org/10.48550/arXiv.0801.3690)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p7.1)\.
- Panko \(2009\)R\.R\. PankoRevising the panko\-halverson taxonomy of spreadsheet risks\.In2009 42nd Hawaii International Conference on System Sciences,Vol\.,pp\. 1–10\.External Links:[Document](https://dx.doi.org/10.1109/HICSS.2009.373),ISSN 1530\-1605Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p2.1)\.
- Panko \(2008\)R\. R\. PankoSpreadsheet errors: what we know\. what we think we can do\.External Links:0802\.3457Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p5.1)\.
- Panko \(2013\)R\. R\. PankoThe cognitive science of spreadsheet errors: why thinking is bad\.In2013 46th Hawaii International Conference on System Sciences,Vol\.,pp\. 4013–4022\.External Links:[Document](https://dx.doi.org/10.1109/HICSS.2013.513)Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p2.1)\.
- Panko \(2015\)R\. R\. PankoWhat we don’t know about spreadsheet errors today: the facts, why we don’t believe them, and what we need to do\.InProceedings of the EuSpRIG 2015 Conference “Spreadsheet Risk Management”,Cited by:[§1](https://arxiv.org/html/2608.26345#S1.p1.1),[§4\.5](https://arxiv.org/html/2608.26345#S4.SS5.p1.1),[§6](https://arxiv.org/html/2608.26345#S6.p1.1)\.
- Parr \(2024\)T\. ParrANTLR\(Website\)External Links:[Link](https://www.antlr.org/)Cited by:[Figure 3](https://arxiv.org/html/2608.26345#S2.F3),[Figure 3](https://arxiv.org/html/2608.26345#S2.F3.5)\.
- Reinhardt and Pillay \(2004\)T\. Reinhardt and N\. PillayAnalysis of spreadsheet errors made by computer literacy students\.InIEEE International Conference on Advanced Learning Technologies, 2004\. Proceedings\.,Vol\.,pp\. 852–853\.External Links:[Document](https://dx.doi.org/10.1109/ICALT.2004.1357687),ISSNCited by:[§6](https://arxiv.org/html/2608.26345#S6.p3.1)\.
- Scaffidiet al\.\(2005\)C\. Scaffidi, M\. Shaw, and B\. MyersEstimating the numbers of end users and end user programmers\.In2005 IEEE Symposium on Visual Languages and Human\-Centric Computing \(VL/HCC’05\),Vol\.,pp\. 207–214\.External Links:[Document](https://dx.doi.org/10.1109/VLHCC.2005.34)Cited by:[§1](https://arxiv.org/html/2608.26345#S1.p1.1)\.
- Schalkwijket al\.\(2016\)S\. Schalkwijk, F\. Hermans, M\. van der Ven, and H\. DuitsAuditing spreadsheets: with or without a tool?\.pp\.\.Cited by:[§6](https://arxiv.org/html/2608.26345#S6.p4.1)\.
- Ziemannet al\.\(2016\)M\. Ziemann, Y\. Eren, and A\. El\-OstaGene name errors are widespread in the scientific literature\.Genome Biology17\(1\),pp\. 177\.External Links:ISSN 1474\-760X,[Document](https://dx.doi.org/10.1186/s13059-016-1044-7),[Link](https://doi.org/10.1186/s13059-016-1044-7)Cited by:[§1](https://arxiv.org/html/2608.26345#S1.p1.1)\.