Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support terminal intelligence #3390

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open

Conversation

life2015
Copy link
Member

@life2015 life2015 commented Mar 11, 2024

终端的智能补全能力

终端的智能补全能力是指终端在输入命令时,能够根据用户的输入,自动提示可能的命令或参数,交互方式类似于编程时的语言服务。

此功能可以增加用户使用终端时的易用性。

image

功能建设

功能建设进度

  • 补全设计优化:目前的设计主要服务于功能验证,因此 UI 看起来很简陋,需要做后续的优化
  • 补全交互方式优化:比如说 上下键选择,Tab 确认。或者 Tab 或者 上下键 选择,Enter 确认
  • Generator 补全支持。
  • 渲染方式优化,目前是直接渲染在 Xterm.js Decorations 上面的,考虑做一个全局 DOM,然后通过 DOM Align + Xterm.js Decoration 来做生命周期绑定和位置绑定
    • 渲染体验进一步优化
  • 代码架构优化
    • 面向对象化,修改 runtime 和 Suggestion 核心逻辑
    • 抽象到 common,然后在 Node.js 层引入,实现部分逻辑覆写(这样方便后续的纯前端场景)
    • Node.js 层逻辑覆写
  • CodeStyle 处理,目前没有对从 inShellisense 项目的代码做处理,考虑到未来比较方便更新代码,不过这块要看看是不是要格式化一下代码什么的
  • 快捷的功能开启关闭 UI

Types

实验性支持了终端的智能补全,体验打磨中。

  • 🎉 New Features

Background or solution

目前 Warp, FIg 这类终端智能产品越来越多,相关的用户心智和社区基建也愈发成熟。
并且在前段时间我们完成了对 Xterm.js Decoration 相关技术的掌握,因此基于 Xterm.js Decoration 和 inShellisense 项目的 Runtime 做出了 OpenSumi 上终端智能的实现。

Changelog

  • feat: (alpha) support terminal intelligence

@life2015 life2015 requested review from erha19, bytemain and Aaaaash March 11, 2024 02:57
Copy link

railway-app bot commented Mar 11, 2024

This PR was not deployed automatically as @life2015 does not have access to the Railway project.

In order to get automatic PR deploys, please add @life2015 to the project inside the project settings page.

@opensumi opensumi bot added the 🎨 feature feature required label Mar 11, 2024
@erha19
Copy link
Member

erha19 commented Mar 11, 2024

讨论是否需要转移补全逻辑到 Node.js

@life2015 个人感觉补全逻辑不应该挪到 NodeJS,相应的数据逻辑倒是可以灵活有点,一个实际状况是这种补全能力,在纯前端上面应该是个痛点,在本地场景,可能还会与系统的一些 GhostText 补全(例如安装了 zsh)冲突

可以基于 WebC 搞个版本,接入 AI 的上下文补全来搞一下?

@life2015
Copy link
Member Author

life2015 commented Mar 11, 2024

讨论是否需要转移补全逻辑到 Node.js

@life2015 个人感觉补全逻辑不应该挪到 NodeJS,相应的数据逻辑倒是可以灵活有点,一个实际状况是这种补全能力,在纯前端上面应该是个痛点,在本地场景,可能还会与系统的一些 GhostText 补全(例如安装了 zsh)冲突

可以基于 WebC 搞个版本,接入 AI 的上下文补全来搞一下?

是的,在纯前端场景上也能享受到终端智能补全能力是我把核心 Runtime 放到前端的主要原因。当时参考的 Fig 以及 inShellisense 都是基于 Node.js 来跑的,我做了一些前端的迁移和适配,理论上把 FS 读取和 Fig Generators 对 bash 的调用做后端代理之后,前后端的功能差异不会很大。

其实对于 OpenSumi 完整版而言,后端实现会更容易,前段实现难一些,不过目前前端实现我也已经跑通了,就是需要再衡量一下两条路线,我个人也是倾向于前端方案,这样会方便未来一些纯前端场景的接入。

目前来看,支持 WebC 的问题不大。我们后续需要把前端对后端的依赖层做一个抽象,然后分开传统场景,WebC,以及云终端各种场景做分别的注入。

@erha19 erha19 added the v3 OpenSumi 3.0 label Mar 27, 2024
@life2015 life2015 force-pushed the feat/terminal-intell-alpha branch from d7eb345 to dd4e0a1 Compare May 29, 2024 07:19
@life2015
Copy link
Member Author

全面更新终端补全能力的交互 99ad0be

@life2015 life2015 changed the title feat: (alpha) support terminal intelligence feat: (beta) support terminal intelligence May 30, 2024
@life2015
Copy link
Member Author

Terminal 智能的底层补全逻辑做了大规模重构,核心逻辑放置于 common,环境相关的逻辑可以单独在 Browser 或者 Node.js 层实现。目前是采用了 Node.js 层实现,通过覆写两个 Class 即可做到。

TODO:
一些交互闪烁和布局问题
cwd 感知有问题

Copy link

codecov bot commented Jun 12, 2024

Codecov Report

Attention: Patch coverage is 13.07815% with 545 lines in your changes missing coverage. Please review.

Project coverage is 54.96%. Comparing base (ef65ac4) to head (5be010d).

Files Patch % Lines
...ackages/terminal-next/src/common/intell/runtime.ts 6.89% 113 Missing and 49 partials ⚠️
...ages/terminal-next/src/common/intell/suggestion.ts 11.23% 114 Missing and 44 partials ⚠️
...rminal-next/src/node/terminal.intell.enviroment.ts 18.00% 35 Missing and 6 partials ⚠️
...ckages/terminal-next/src/common/intell/template.ts 12.50% 32 Missing and 3 partials ⚠️
packages/terminal-next/src/common/intell/parser.ts 10.52% 21 Missing and 13 partials ⚠️
...minal-next/src/node/terminal.intell.spec.loader.ts 8.10% 27 Missing and 7 partials ⚠️
...kages/terminal-next/src/common/intell/generator.ts 20.51% 21 Missing and 10 partials ⚠️
...ages/terminal-next/src/common/intell/utils/ansi.ts 0.00% 15 Missing and 8 partials ⚠️
...erminal-next/src/node/stupid-shell-intergration.ts 0.00% 9 Missing ⚠️
...-next/src/browser/contribution/terminal.command.ts 0.00% 7 Missing and 1 partial ⚠️
... and 4 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3390      +/-   ##
==========================================
- Coverage   55.25%   54.96%   -0.29%     
==========================================
  Files        1545     1557      +12     
  Lines       94179    94806     +627     
  Branches    19331    19532     +201     
==========================================
+ Hits        52035    52114      +79     
- Misses      34978    35384     +406     
- Partials     7166     7308     +142     
Flag Coverage Δ
jsdom 50.39% <0.15%> (-0.34%) ⬇️
node 15.52% <13.07%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@life2015 life2015 changed the title feat: (beta) support terminal intelligence feat: support terminal intelligence Jun 13, 2024
left: 0;

// TODO 考虑动态设置字体
font-family: Menlo, Monaco, 'Courier New', monospace;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

调整成 opensumi 全局用的 font-family 吧,代码库里搜一下


private initContainer() {
this.popupContainer = document.createElement('div');
this.popupContainer.style.zIndex = '12';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要 hard code,记在 StackingLevel 这个枚举值里

this.listenPromptState(client.term);
} catch (e) {
// eslint-disable-next-line no-console
console.error('listenTerminalEvent', e);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

使用 this.logger

private getConnection(xterm: Terminal): ITerminalConnection {
// @ts-ignore
// 目前需要强制取用 Addon 的 connection 能力
const attachAddon = xterm._addonManager._addons.find((addon) => !!addon?.instance?.connection);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CleanShot 2024-06-18 at 17 03 37@2x

attachAddon 里就有 connection, attachAddon 可以在 TerminalClient 里拿到。

可以看看 https://github.com/opensumi/core/pull/3390/files#diff-ffe2330864e5dbde73f2b9622b05274eaf2f8d98cd7daea1d80277e8be4f8a48R72

TerminalClient 是一个 xterm 渲染的起始位置,肯定是可以拿到这个 attachAddon 的

@erha19 erha19 self-assigned this Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🎨 feature feature required v3 OpenSumi 3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants