=PY() + NumPy inside Collabora

[KeithCu]

Dear Collabora-Dev,

I’ve been working (with others) on a Python / UNO extension (currently targeting LibreOffice) to enable AI features as well as NumPy. The extension supports a new =PY() function, an optional shared kernel between cells, various NumPy-based analysis tools, color-syntax editor, and OCR.

I recently broke out the =PY() and the related functionality into a subset extension called LibrePy. Does Collabora still support Python / UNO? Is this something you would be interested in including?

The main dev doc is here: Enabling_Numpy

Regards,

-Keith

[darshan]

Hi Keith,

@KeithCu

I’ve been working (with others) on a Python / UNO extension (currently targeting LibreOffice) to enable AI features as well as NumPy. The extension supports a new =PY() function, an optional shared kernel between cells, various NumPy-based analysis tools, color-syntax editor, and OCR.

I recently broke out the =PY() and the related functionality into a subset extension called LibrePy

Thanks a lot for sharing this — LibrePy looks like a really interesting project, and your docs on the NumPy/ABI problem and the venv-subprocess design are a genuinely great read!

To answer your question: yes, Python/UNO is still supported in Collabora technology, but the picture differs quite a bit depending on which product we’re talking about, so let me break it down.

Collabora Office Classic (our LibreOffice-based desktop suite, using the traditional VCL toolkit) has full Python/UNO support — PyUNO bridge, the extension framework, macro editing, all of it works essentially the same as in LibreOffice. LibrePy should be usable there largely as-is.

The new Collabora Office for desktop (released November 2025) is a different architecture — it brings the Collabora Online experience to the desktop, with the UI built on web technologies (JavaScript, Canvas, WebGL) rather than VCL. Macro support there is limited to executing existing scripts; there’s no full macro editor or advanced Python/UNO tooling, and desktop-style extension UI doesn’t apply. For advanced macro work we point people to Classic.

Collabora Online (server) is the most constrained. Macro execution — including Python scripts — is disabled by default for security and has to be explicitly enabled by the admin in coolwsd.xml (enable_macros_execution). Scripts generally need to be baked into the server image, and each document runs inside a locked-down jail that cannot call external programs, use the shell, or access the wider filesystem. That’s the key blocker for LibrePy’s current design: the warm venv-subprocess worker and the pywebview/Monaco editor window rely on exactly the things the jail forbids, and desktop windows can’t render in a browser canvas anyway.

So in short: feasible on Classic today, but the =PY() execution backend would need a fundamentally different, server-safe design for Online and the new desktop app (which share the same codebase — so a rework would benefit both at once).

As for inclusion — I can’t speak for the product roadmap myself, but the best way to start that conversation is to open a discussion or enhancement request on the Collabora Online GitHub repo (GitHub - CollaboraOnline/online: Issue tracker only. Active development is on Gerrit at https://gerrit.collaboraoffice.com/. · GitHub) with a pointer to your design docs. Framing it around what a jail-safe execution model could look like, rather than the desktop venv approach, would make it much easier to evaluate.

Thanks again for reaching out — it’s genuinely exciting to see this kind of experimentation around Python in Calc!

Cheers
Darshan

[KeithCu]

Hi Darshan,

Thank you for the encouragement and explanation. I spend time on docs so I’m glad you noticed.

I opened an enhancement request framing a jail-safe design along the lines you suggested: Jail-safe scientific Python =PY() for Calc via a coolwsd-brokered compute service · Issue #16010 · CollaboraOnline/online · GitHub

I started work on a basic compute service: writeragent/compute_service at master · KeithCu/writeragent · GitHub

I’m not sure of your scaling strategy so it’s single threaded for now. NumPy releases the GIL lock for many operations so even multiple threads could be somewhat useful. Multiple processes are more scalable but it’s another hop so I avoided it for now.

I’m working on a prototype version of the C+±side to call the Python compute service. It’s building.

Keith

[KeithCu]

Hi,

I’ve got an =PY() implementation that runs in Collabora Online and seems close to being ready to submit to Gerrit, but I wanted to discuss a few things before we go too far.

1. This design treats PY() as a spreadsheet formula with explicit dataflow. Conceptually it is PY(code, data…): the Python code (or a cell reference to shared code) is one argument, spreadsheet ranges are passed explicitly as formula arguments, which define the dependencies in the Calc DAG. For usability, single-row and single-column inputs are normalized to 1D data, while true rectangular blocks remain 2D. The script publishes its output through result = …, and the returned value is mapped back into Calc scalars or spill ranges.

Microsoft’s =PY() is closer to an embedded Python cell than to a conventional spreadsheet function. Its public syntax is =PY(python_code, return_type), where python_code is static text and return_type selects either an Excel value projection or a Python object result. Spreadsheet data is typically accessed indirectly from inside the Python code via xl(…), rather than through explicit formula arguments. As a result, dependency discovery and recalculation semantics are tied to Excel’s Python integration layer rather than to ordinary formula argument flow.

There is also a scheduling difference. Our model is intended to follow normal spreadsheet dependency semantics: if a Python cell depends on A1:B10, that dependency is visible in the formula signature itself. Microsoft’s Python-in-Excel model has specialized worksheet behavior, including row-major evaluation of Python cells, which is not the same as simply adding another pure function node to the formula graph.

The result model is also different. Our contract is assignment-oriented (result = …) and aimed at producing Calc-compatible scalar or matrix values. Microsoft’s model distinguishes between “Excel value” and “Python object” return modes, which means the cell may either materialize a grid/spillable Excel value or hold an object-backed Python result for downstream Python use.

The benefit of the xl() function is that it’s a pain to pass in lots of parameters, but you can mirror various random config values into a range, and just pass that range in.

2. Another question has to do with the namespace. Right now the =PY is in the org.extension.writeragent.PythonFunction namespace, but for the C++ code I currently have: com.sun.star.sheet.addin.PythonComputeFunctions.getPy. Should it be ORG.OPENOFFICE.PY?

3. Another issue has to do with the server side. I’ve got the C++ side working but the compute service side is a Python HTTP listener which calls into the existing plugin routines for the AST sandbox, auto-imports, various helper routines, etc. LibrePy.oxt is 27 kloc, plus 20 kloc of test code for scripting + Calc , and it’s not clear how much of it you will want in your builds. Maybe you could keep the compute service in your tree (which is small and security related) and just pull the rest from my tree for your builds? I evolve it constantly still and I’d rather not have to go through a huge process to push new versions. You would want a different config / location for the parameters, but otherwise the code should be usable as-is.

I’d appreciate some advice!

Regards,

-Keith

[mmeeks]

Hi Keith,

First - thanks for your work here! Looks very interesting.

I’ve got an =PY() implementation that runs in Collabora Online and seems
close to being ready to submit to Gerrit, but I wanted to discuss a few
things before we go too far.

I expect Quikee will ultimately want to review this.

1. This design treats PY() as a spreadsheet formula with explicit
   dataflow. Conceptually it is PY(code, data…): the Python code (or a
   cell reference to shared code) is one argument, spreadsheet ranges
   are passed explicitly as formula arguments, which define the
   dependencies in the Calc DAG. For usability, single-row and single-
   column inputs are normalized to 1D data, while true rectangular
   blocks remain 2D. The script publishes its output through result =
   …, and the returned value is mapped back into Calc scalars or spill
   ranges.

That seems like a useful feature; but I expect we will want (for interop. reasons) to make =PY() behave like the Excel one - this brings a number of interesting issues.

In particular that from a quick skim of the docs - it seems like Microsoft builds all your python into (effectively) a single script that executes in Excel’s defined calculation order: left-to-right,top-to-bottom - and we are dramatically more flexible in terms of the calculation graph.

Microsoft’s =PY() is closer to an embedded Python cell than to a
conventional spreadsheet function. Its public syntax is =PY(python_code,
return_type), where python_code is static text and return_type selects
either an Excel value projection or a Python object result. Spreadsheet
data is typically accessed indirectly from inside the Python code via
xl(…), rather than through explicit formula arguments. As a result,
dependency discovery and recalculation semantics are tied to Excel’s
Python integration layer rather than to ordinary formula argument flow.

Right; so it’s necessary to re-calculate everything python in one shot I think (?); that is pretty weird to a spreadsheet-guy but perhaps there is some sensible rationale around data-frames, pandas type stuff and so
on: not entirely clear. Its not entirely clear to me what happens if other cells depend on python results - it would (really!) suck to have to sort our calculation
order into excel-order and block unrelated calculations on the results
of PY().

There is also a scheduling difference. Our model is intended to follow
normal spreadsheet dependency semantics: if a Python cell depends on
A1:B10, that dependency is visible in the formula signature itself.
Microsoft’s Python-in-Excel model has specialized worksheet behavior,
including row-major evaluation of Python cells, which is not the same as
simply adding another pure function node to the formula graph.

I must confess that reading the docs on the Python integration feature in Excel looks extremely contrived, different and looks like a classic of product graunching. Still I’ve not really had much time to think through why it might be useful to do this.

The result model is also different. Our contract is assignment-oriented
(result = …) and aimed at producing Calc-compatible scalar or matrix
values. Microsoft’s model distinguishes between “Excel value” and
“Python object” return modes, which means the cell may either
materialize a grid/spillable Excel value or hold an object-backed Python
result for downstream Python use.

Right; putting a new set of complex data-types eg. data-frame into calc formula results is ‘fun’

The benefit of the xl() function is that it’s a pain to pass in lots of
parameters, but you can mirror various random config values into a
range, and just pass that range in.

Another question has to do with the namespace. Right now the =PY is
in the org.extension.writeragent.PythonFunction namespace, but for
the C++ code I currently have:
com.sun.star.sheet.addin.PythonComputeFunctions.getPy. Should it be
ORG.OPENOFFICE.PY?

Great question; I’d leave to quikee; org.collaboraoffice. is a namespace we have a moral claim on though.

3. Another issue has to do with the server side. I’ve got the C++ side
   working but the compute service side is a Python HTTP listener which
   calls into the existing plugin routines for the AST sandbox, auto-
   imports, various helper routines, etc. LibrePy.oxt is 27 kloc, and
   it’s not clear how much of it you will want in your builds. Maybe
   you could keep the compute service in your tree (which is small and
   security related) and just pull the rest from my tree for your
   builds? I evolve it constantly still and I’d rather not have to go
   through a huge process to push new versions. You would want a
   different config / location for the parameters, but otherwise the
   code should be usable as-is.

I’m really rather curious how this works; clearly we have an in-process python interpreter that we can rely on in CODA and also COOL (when enabled etc.) and we’d want to re-use that wherever possible. I’d appreciate some advice! more poisonous excel mis-features, calculation order, inter-PY()
invocation variable persistence (perhaps better called one big
script’iness and so on.

If we want to do something nicer with a custom =PY_CO() type thing we should think that through carefully of course so we don’t make such a
mess for the future ourselves. But I expect what I’m missing is the way
that data-analysists got used to using pandas and iterating / stepping
through large complex scripts - there must be some benefits to what they
have done.

Thoughts much appreciated but I’d love to see some twisted xlsx

using pathological =PY() stuff, so we can estimate the impact of forcing
calculation ordering etc.

Thanks for digging deeply into this !

Best regards,

Michael. (who annoyingly had to edit his indentation on-line to get discord to do something sensible here.)

[KeithCu]

Hi Michael,

Glad to hear you are interested in this side project of WriterAgent

It’s generally worth following MS for compatibility but here I think they went off a cliff. The smart people who created Excel are long gone so I wouldn’t trust the new group as much, and they have big perf challenges with the constant recalcing.

However, the primary reason I sent these emails is to get input on this difficult, key topic so I look forward to what you come up with. I’ve only written test spreadsheets, I haven’t tried real problems yet.

You will also need to find someone with more Calc knowledge to do their design because it’s easy to write a new C++ PY() function in the LO/Collabora codebase that marshalls via HTTP to a LibrePy-based compute service, but changing the recalc engine and the other stuff required, and then making it fast is another matter.

Another possibility is to do the simple =PY() and then when you have free time, implement theirs and rename it as PY_XL() so there is no conflict.

It would be easy in Python to rewrite their scripts on import and remove their xl() calls and change them to argument parameters for better fidelity with XLSX. The reverse is also generally doable, so I’d consider these hacks before doing their design.

For the server, it doesn’t use the “standard” Python interpreter since it may not have NumPy. Also, it seems better to run user NumPy out of proc since it is a big code footprint and can use lots of ram, hang, crash, etc. LibrePy by default spins up multiple subprocesses for core, OCR and vector search (in WriterAgent), so that they don’t fight for the GIL and there is better robustness and memory locality for the different workloads.

LibrePy has a fast IPC but it needs Python on both sides for now so this one uses ye olde JSON to marshall from C++ to the Python service.

The initial design based on input was to run the server.py from a configured venv, easily callable from C++. The current code is a threadpool http handler which calls into the existing LibrePy code for everything and returns it as JSON to the C++. The compute service is in WriterAgent/compute_service. (direct GH links seem disallowed.)

I’d be grateful for feedback on the initial compute service and anything else!

-Keith

[KeithCu]

The =PY() code is working (for scalar return values) but the error UX is a little confusing.

What happens today:

• Interim: cell shows the string #BUSY! (not a FormulaError; there is no FormulaError::Busy).

• Feature turned off: string #DISABLED (so it’s obvious without digging in logs).

• Most failures (Python exception, bad JSON, many coolwsd/network errors): FormulaError::NoValue → #VALUE!.

• Timeout / no emitter / superseded pending: FormulaError::NotAvailable → #N/A.

What we think we know about Calc

1. True formula errors (CreateDoubleError / void) keep ISERROR / IFERROR and the Online help button, but only fixed short/long strings.

2. Custom readable text in the cell means finishing the volatile with an OUString (same pattern as #BUSY! / #DISABLED), which is not a formula error.

So it seems we can’t both keep a pure FormulaError cell and show SyntaxError: unexpected EOF without inventing a new Core/Online channel.

Questions:

Given it’s almost impossible to figure out what has gone wrong when you just see #VALUE!, since it could be any of countless errors, is treating all Python failures as a short string (like #DISABLED) acceptable so authors can see the cause, accepting that those cells won’t be ISERROR?

Or do you prefer keeping only FormulaError (#VALUE! / #N/A) and treating logs or some other existing UI as the place for the message?

Any advice appreciated!

-Keith

[mmeeks]

Hi Keith,

The =PY() code is working (for scalar return values) but the error UX is
a little confusing.

Heh; so - I guess we can already include custom python formulae into
calc’s formula namespace and map parameters to the function parameters,
I’m curious what adding =PY() adds as a benefit there - is it returning
new types or ? …

What happens today:
*
Interim: cell shows the string |#BUSY!| (not a |FormulaError|; there
is no |FormulaError::Busy|).

Yep; so - looking at Microsoft’s =PY() it seems there are a set of new
return values of python types eg. you can return a data-frame - we
really need to work out how those intersect with others - since
expanding the set of types that can be returned to the core is something
of a nightmare.

What is a dataframe &* a boolean (for example) - note we don’t currently
have native boolean types in CO calc - where Excel does.

What we think we know about Calc
1.
True formula errors (|CreateDoubleError| / void) keep |ISERROR| / |
IFERROR> and the Online help button, but only fixed short/long strings.

Yes - ideally we could improve our error handling in calc anyway - but
it badly needs careful thought.

Just to underline this again - any change to the calc core needs to be
supported indefinitely (forever) - so there is nearly an unbounded
future cost to the change there in maintenance, and in interoperability.
Sorry if it seems painful that we try to do this carefully - and also
compatibly but … we do

Also sorry for the slow reply - Quikee is prolly the lead reviewer here
and is on vacation.

2. Custom readable text in the cell means finishing the volatile with
   an |OUString| (same pattern as |#BUSY!| / |#DISABLED|), which is /
   not/ a formula error.

So it seems we can’t both keep a pure |FormulaError| cell /and/ show |
SyntaxError: unexpected EOF| without inventing a new Core/Online channel.

Right.

Questions:

Given it’s almost impossible to figure out what has gone wrong when you
just see #VALUE!, since it could be any of countless errors, is treating
all Python failures as a short string (like |#DISABLED|) acceptable so
authors can see the cause, accepting that those cells won’t be |ISERROR?|

So - one thing we could do in the core, without changing the
formula-language is have some look-aside buffer of more detailed errors
for specific locations that could be interrogated. This of course could
bring a whole host of performance problems clearer errors in the error
→ no error transition, but we can probably safely assume that not
having errors is by far the most common case

Or do you prefer keeping only FormulaError (|#VALUE!| / |#N/A|) and
treating logs or some other existing UI as the place for the message?

Logs are cheap of course, but where do you see them. We’re not great at
having a ‘console’ on the side of the codebase that you can see output
and provide commands to help debug things - probably with COOL/CODE and
a browser front-end we could get better at that if we tried.

Any advice appreciated!
Hope this helps let me get to your earlier mail.

Regards,

Michael (whose E-mails are typically mangled by discord sorry!)

[KeithCu]

Hi Michael,

While custom UNO Add-ins register fixed schema functions into Calc’s formula namespace, it’s a manual process. =PY() adds flexibility by allowing ad-hoc code execution, Excel spreadsheet interoperability, and rich stateful data objects like pandas DataFrames or plots rather than just standard UNO scalars/arrays. The good news is it seems there currently is no need to bloat ScFormulaCell with Python types. =PY() can spill DataFrames as standard 2D cell arrays, charts as images, etc. keeping state in the venv worker.

Also the implementation runs code out-of-process allowing users to leverage heavy scientific stacks and specialized workloads (OCR, FTS / embeddings) without an ABI mismatch risk or crash to the main process.

Regarding errors, I just added a diagnostics sidebar to LibrePy recently. What about something like that?

However, I don’t know your TS stack and don’t want the MVP to be bigger than necessary.

I said it was “easy” to convert the MS xl() syntax to the standard format, but that’s only true for static cases like xl(“A1”). However, that seems to be what I mostly find on the web. I have a prototype implementation that removes the XL() calls and converts to the =PY(code, data) on import and converts back on export. It converts the script syntax and the disk format. I wanted to post a GitHub link but it got blocked.

It’s gotten somewhat complicated but it’s far less than implementing xl() and all the other features natively. I found hooks that should work magically behind the scenes for LO / Classic, but getting that implemented in Collabora Online requires C++ changes if you want to go that route.

Besides recalc, I think it’s bad they store all the Python in non-cells. I like the idea of moving beyond OOP to cell-oriented programming.

Thanks!

-Keith

[mmeeks]

Hi Keith,

It’s generally worth following MS for compatibility but here I think
they went off a cliff. The smart people who created Excel are long gone
so I wouldn’t trust the new group as much, and they have big perf
challenges with the constant recalcing.

Yes, of course the problem is that no matter what mess has been made,
ultimately we will need to interoperate with it forever.

Then again - sleeping on the constraints; I think we could actually do
this quite easily - we special case =PY() functions, and we add
dependency ranges of all the formulae above and to the left of the cell
in the sheet (and perhaps preceeding sheets). That means we should get
the calculation order right just for these cells.

However, the primary reason I sent these emails is to get input on this
difficult, key topic so I look forward to what you come up with. I’ve
only written test spreadsheets, I haven’t tried real problems yet.

Oooh =) can you share your Excel =PY() torture-test sheets ? we could
drop those into git straight away as future unit tests.

As well as getting the function references right, intersecting the
various types is helpful with operators, eg. this is a great sample:

Client Challenge

Have a look through the sheets there if we add new types we need to
know the impact.

You will also need to find someone with more Calc knowledge to do their
design because it’s easy to write a new C++ PY() function in the LO/
Collabora codebase that marshalls via HTTP to a LibrePy-based compute
service, but changing the recalc engine and the other stuff required,
and then making it fast is another matter.

So - I think the recalc thing is actually fairly easy to get the
ordering - cf. my other mail.

Another possibility is to do the simple =PY() and then when you have
free time, implement theirs and rename it as PY_XL() so there is no
conflict.

I’d prefer to have =PY() be the compatible one; ultimately we can call
python functions in formulae anyway - quite possibly we could just add
some nice UI to make that easier to create and reference python formulae
in cells ? In COOL it would be great to have some pretty code-editor in
a side-bar for those simple things too I guess - that connects nicely to
those cells: possibly we can do much of this in the UI (?)

It would be easy in Python to rewrite their scripts on import and remove
their xl() calls and change them to argument parameters for better
fidelity with XLSX. The reverse is also generally doable, so I’d
consider these hacks before doing their design.

Hmm; I guess xl() can have functions passed in to construct references
as strings (?) which always tends to be the problem with over-powerful
reference things. I wonder if there is a defined behavior of having old
cell values for things after the =PY() function in the calc order - so
you can do awful iteration things there =) would be nice not to have
that ordering constraint.

For the server, it doesn’t use the “standard” Python interpreter since
it may not have NumPy. Also, it seems better to run user NumPy out of
proc since it is a big code footprint and can use lots of ram, hang,
crash, etc. LibrePy by default spins up multiple subprocesses for core,
OCR and vector search (in WriterAgent), so that they don’t fight for the
GIL and there is better robustness and memory locality for the different
workloads.

Oh; hmm; so - I guess for COOL we can have these spawned from the forkit
somehow I guess.

LibrePy has a fast IPC but it needs Python on both sides for now so this
one uses ye olde HTTP to marshall from C++ to the Python service.

No shame in that

The initial design based on input was to run the server.py from a
configured venv, easily callable from C++. The current code is a
threadpool http handler which calls into the existing LibrePy code for
everything and returns it as JSON to the C++. The compute service is in
WriterAgent/compute_service. (direct GH links seem disallowed.)

It is interesting; do we know if Excel does the same ? (it looks like
it) - if so, good - and then it might be interesting to see what
protocol and what contents they use between their processes: that can
perhaps show us if they build a single, larger python app, parse it as a
one-shot and then invoke sub-functions.

I’d be grateful for feedback on the initial compute service and anything
else!

Sounds very interesting thanks so much for digging into all of this.

My first instinct in quikee’s absence is either to start with defining
what we want to do interop-wise by torturing testing Excel and building
some samples as a goal to ‘fix’ - or - by just making python easier to
use by improving the calc / python UX - which we can do without adding a
new function I think (?).

Hope that helps,

Michael.

[KeithCu]

Hi Michael,

On Excel interop, where I’ve landed so far is doing the cheap half of compatibility: reading / writing the on-disk format, and the expensive half is copying their PY scheduler into Calc.

I was mistaken earlier, what Excel saves after edit is closer to the current LibrePy design than I initially realized: static xl(“A1:B10”) gets rewritten to placeholders in pythonScripts.xml, and the cell formula is _xlws.PY(index, returnType, …deps) with the ranges as real trailing formula arguments.

LibrePy now has a bidirectional OOXML rewriter that maps the code into cells and converts native =PY(code; ranges…) to the MS format (and back on export). It seems true dynamic xl(variable) is not a Microsoft product path yet so we can ignore for now.

I don’t have torture sheets, just a few samples I found on GitHub (GitHub - dbogt/PythonExcel: Demos of Python in Excel · GitHub) to test the round-trip script. My current proposal is about a 4000 line patch to lokit and online that calls into Python /LibrePy for everything including conversion.

On calculation order / “depend on everything above and to the left”: it’s a small Core idea, but it under-approximates Excel and over-taxes Calc. Excel’s co-volatility re-runs all PY cells when any PY is dirty (row-major), plus the Excel↔Python flip-flop.

Flip-flop is the extra scheduler needed when normal Excel formulas and PY cells depend on each other in a chain. Excel and Python use incompatible calc models, so Excel does not fold PY into one dependency pass. It alternates:

1. Run an Excel pass on dirty non-PY cells (dependency order, partial).
2. When a cell needs a PY result that is not ready yet, defer that cell (push it later in the chain).
3. Run a Python pass: because of co-volatility, that means all PY cells (LTR / TTB).
4. Resume Excel for cells that were waiting on those PY results.
5. If another PY still depends on a newly computed Excel cell, defer again and do another full PY pass.
6. Repeat until nothing dirty remains.
   Synthetic above/left deps would force ordering among cells that recalculate, but they also create a lot of false dependencies, and on Online that multiplies remote POSTs to the compute service. In spite of the bad performance, it still wouldn’t be fully compatible.

Regarding the Microsoft backend: Excel also runs scientific Python out of the spreadsheet process: hypervisor-isolated Azure containers with a curated Anaconda image, reached over Microsoft’s OfficePy HTTPS control plane (Bearer, runtime setup, then code + cell data).

Public reverse-engineering and process lists point at a Jupyter-like kernel per workbook. Behaviorally it looks like a shared kernel with sequential cell execs into globals (plus co-volatility). Our compute_service (stdlib HTTP, POST /v1/execute, optional shared session) seems the Collabora-shaped version of that split.

On the “we can call Python from formulae / nicer UDF UI” idea: that path is real and worth improving, but it solves a different problem than =PY().

Stock Calc can reach Python only as named functions. That’s a catalog API. A COOL sidebar that helps create and insert those names would be lovely, and Classic LibrePy has Monaco + a Python sidebar for editing, but the execution surface is still “call one of the known functions.”

Excel Python (and what data analysts paste into cells) is ad-hoc pipelines, not a fixed API. A cell might be “load this range as a DataFrame → dropna → groupby Region → mean of Sales → return,” and the next cell a completely different pandas/numpy/matplotlib story. Those scripts live as snippets.

Without a generic Add-In that accepts code plus explicit data ranges (=PY(code, data…)), you’d need either a new Calc function per cell or you’d reinvent “run this string” under another name.

I’d treat nicer UDF/editor UX as complementary for curated helpers, and keep =PY(code, data…) as the surface that can import Excel sheets and run real scientific pipelines.

BTW, besides =PY / xl() / co-volatility, Excel Python workbooks also can use package and formula features Calc can’t evaluate: Table[#All], ANCHORARRAY/spill parents, engine spill, returnType object cards, and long scripts vs Calc’s ~1024 formula MAXSTRLEN.

We already treat those on file rewrite: on open we snapshot tables/spill parents to A1, store long code on py_code sheets, map to DAG =PY, and strip Excel Python package parts; on save we restore pythonScripts.xml, _xlws.PY, and the original Table/ANCHORARRAY tokens from side meta. That’s round-trip fidelity, not live Calc Table/spill support. Growing tables and live spill parents won’t update until the engine adds those features.

It seems Calc’s strength is the dependency DAG and partial recalc. For the optional shared kernel mode, LibrePy requires user pass upstream cells as data so the DAG orders precedents, rather than re-running the world.

It’s gray area to talk about how well import / export works. It could be possible to say for v1, that “well-designed” scripts that respect Calc’s dependency order (and engine features) run.

Update: One more point I forgot to go into: right now my patch doesn’t start the compute service yet. I’ll research the forkit code and try to figure out how to use that.

Thank you again for the advice.

Keith

[KeithCu]

Hi Collabora-dev,

I’m excited to say that the MVP is now up on Gerrit:

1. scaddins: add Calc =PY() Python compute AddIn — Core AddIn, XVolatileResult / #BUSY!, dumb JSON ↔ Any, CppUnit

2. kit/wsd: broker =PY() to python compute service — kit emitter + coolwsd http::Session POST (same outbound shape as AI chat), feature flag default off, UnitWSD wire test

The Python compute service must be installed separately for now. Those patches are a good start, here’s my high-level take on future work:

Phase	Goal
0 — MVP	Merge experimental =PY() default-off; publish first compute image
1 — Calc parity	Visible short Python errors; auto-spill; plot insert; optional shared kernel (wsd-owned session_id)
2 — Authoring UX	Browser Monaco / Run Script / diagnostics (not pywebview in the jail)
3 — Excel files	Use LibrePy OOXML rewriter; Online calls convert on open/save (or a thin filter hook)
4 — OCR	Expose LibrePy-style local OCR + config settings UI.
5 — Science	Helper analysis tools, admin-baked domain stacks in the image

Here’s a list of engine features that unlock Excel parity:

1. Auto-spill / dynamic arrays for AddIn PY
   Today a single-cell =PY that returns a list/ScMatrix keeps only the top-left value unless entered as a matrix formula. Classic LibrePy fakes spill with UNO write-back + #SPILL!; that is a poor fit for Online. Prefer promoting PY into Calc’s dynamic-array path so that spill ranges are engine-owned and dependents work.

2. Longer Python in formulas (MAXSTRLEN)
   One formula symbol is still ~1024 chars → Err:513 on real Excel-length scripts inline. We work around with py_code_* sheet banks (>1000 chars). Raising / growing the string-literal limit (toward tens of KB) would retire a lot of that workaround for both Classic and Online.

3. Plot insert
   Service already returns images[]; Core shows a placeholder. Small kit/LOK insert near the formula cell.

4. For Excel fidelity beyond round-trip snapshots
   Live Table / structured refs and ANCHORARRAY/spill-parent refs — today we snapshot to A1 on import so files round-trip, but growing tables and live spill parents need real engine support.

Keith

[mmeeks]

Hi Keith,

I’m excited to say that the MVP is now up on Gerrit:

So - first, I’m sorry it took so long to get to reviewing this - found
some time owing to being on holiday

Big picture overview - I love it; its great to see what you’ve done
here. I particularly like:

* that it remotes the python into another process

* that we have this JSON <-> UNO mapping logic - that
  should be generalized and got in - I'm sure we duplicate
  bits of this around the place and it should be shared
  centrally - I imagine Stephan would be interested in that

* that it is an experimental feature - actually, if we can
  get this in disabled by some configure conditional it might
  be good to get it in now.
	+ amusingly this also 'fixes' CI ;-) but really
	  we should prolly get it to pass CI with it enabled.

The overall design looks ok - I guess there are always some random 

nit-picking.

It would be good to start re-factoring and getting this in -piece-wise, 

Quikee - can you have a look at that ? I suggest - first sharing the UNO
↔ JSON mapping pieces somehow - I think we do a fair bit of that in
core already and should look at sharing it.

Threading wise we should prolly try to be more realistic - we have 

1+epsilon threading protected by the SolarMutex - so, most likely we
should just stick with that at all points that we can.

Naming wise:

engine/scaddins/source/pythoncompute/pythoncompute_anyjson.hxx

We tend not to add pythoncompute_ type prefixes to filenames.

For Excel fidelity beyond round-trip snapshots
Live Table / structured refs and ANCHORARRAY/spill-parent refs —
today we snapshot to A1 on import so files round-trip, but growing
tables and live spill parents need real engine support.

So - I'm excited about this thanks Keith!

Looking really nice; but I'm out for some weeks - hopefully its 

possible to iterate, and re-factor this into something really beautiful.

Would still really like to see some torture test Excel sheets for their 

functionality though !

Good stuff,

	Michael.

[KeithCu]

Hi Michael,

Thank you for the review! I’ve put up a new patchset on Gerrit.

1. Threading — I have changed it so shared AddIn state is SolarMutex-only now. The extra std::mutexes on the pending map / param cache / volatile are gone.

2. Configure gate — add a configure conditional so the AddIn/kit bits can land experimental/disabled at build time as well as runtime (security.python_compute.enable).

3. I fixed the names. Dropped the pythoncompute_ filename prefixes. C ABI names (pythoncompute_set_emitter, etc.) are unchanged.

4. UNO ↔ JSON sharing — I like the goal and would love Stephan’s eye. What I’ve found so far in the tree:

• jsuno appendUnoAsJson / parseJsonToAny — typed (caller must know the UNO Type), nested Any as {“type”,“val”}, NaN throws; lives behind optional QuickJS and isn’t a current scaddins dependency.
• pythoncompute — untyped “dumb JSON” for the compute wire: JSON shape chooses the Any, NaN/Inf → null, 1×N/N×1 flatten, rectangular arrays → sequence<sequence<…>>, plus the execute/result envelope and formula-error / #DISABLED / images policy.
• coolwsd Poco on the broker side.
• tools::JsonWriter — already the shared encoder we use on emit.
• A couple of {type,value}-style helpers (comphelper::JsonToPropertyValues, kit-side unoAnyToJson) that look related but serve PropertyValue / command-arg trees rather than Calc cell grids.

If you or Quikee/Stephan of existing code we overlooked, I’d rather reuse that.

Here are the sheets I’ve been using so far for my round-trip testing, but that might not meet your definition of torture: GitHub - dbogt/PythonExcel: Demos of Python in Excel · GitHub

Keith

[sberg]

I’ve left some rough thoughts in a comment at
https://gerrit.collaboraoffice.com/c/online/+/8122/5#message-6496fd556e5d228cba2b0237c697ec9859179466
“scaddins: add Calc =PY() Python compute AddIn”. Let me know if you’d
want to go that route of reusing what we already have (which I’d suggest
we should try to do), and I can prepare the existing jsuno.hxx to be
reusable for your use case.

[KeithCu]

Hello Stephan,

I’d be happy to re-use existing code to make the patch smaller. Please go ahead and I’ll build on top.

One question: after we strip the envelope we still don’t have a UNO Type for parseJsonToAny (42 / “hi” / [1,2,3]). jsuno’s TypeClass_ANY {“type”,“val”} isn’t what the service sends. If you can add an untyped JSON→Any, we can wrap that; otherwise we’ll keep the hand parser for results and reuse json.hxx on encode first.

Thank you!

-Keith

[sberg]

Hi Keith,

I’ve set you as reviewer on
https://gerrit.collaboraoffice.com/c/online/+/9575 “Move
jsuno-internal json.hxx to comphelper/json.hxx” and
https://gerrit.collaboraoffice.com/c/online/+/9576 “Introduce
comphelper::parseJsonToInferredAny”, so we can discuss any further
details directly in those Gerrit changes.