BIBO backend service - Printable Version

BIBO backend service - Printable Version

+- Rigorous and Reliable (RAR) (http://rar.shufangkeji.com:60380)
+-- Forum: 大计算 (/forumdisplay.php?fid=257)
+--- Forum: 应用 (/forumdisplay.php?fid=363)
+---- Forum: BIBO (/forumdisplay.php?fid=602)
+---- Thread: BIBO backend service (/showthread.php?tid=9169)

BIBO backend service - lingu - 09-26-2020 03:54 PM

We should make BIBO backend a service.

An example of starting sequence for the backend is in http://rar.shufangkeji.com:60380/showthread.php?tid=9169&pid=43605#pid43605

BIBO后台系统的设计应考虑高并发访问，并提供几乎不间断服务。

为此我们需要多个BIBO后台协同运行，也就是需要多个Data Thinker系统协同。为此我们扩展Data Thinker为CODA -- http://tab.d-thinker.org/showthread.php?tid=16288

客户端可访问任何一个访问点进行查询。

---
20201006/zhihao: add D info

RE: BIBO backend service - lingu - 09-26-2020 03:55 PM

Stop and start sage for bibo.

Code:

[sage2@g1b-6 sage]$ bin/sage stop

Sage is stopped.

[sage2@g1b-6 sage]$ bin/sage start

Trying to start Sage ....

Sage is started.

[sage2@g1b-6 sage]$

Start simpleserv.

Code:

[zhihao@g1b-2 bibo]$ pwd

/thinker/globe/soft/bibo

[zhihao@g1b-2 bibo]$ ./simpleserv.sh

RE: BIBO backend service - lingu - 09-29-2020 09:56 AM

zhihao - pls link this to bibo D info -- http://rar.shufangkeji.com:60380/showthread.php?tid=9162

Pls also create a new CT for the bibo backend.

RE: BIBO backend service - lingu - 10-06-2020 11:49 AM

(09-29-2020 09:56 AM)lingu Wrote: Pls also create a new CT for the bibo backend.

Created?

RE: BIBO backend service - lingu - 10-06-2020 11:51 AM

BIBO后台应支持高并发访问，并提供几乎不间断服务。

为此我们需要多个BIBO后台协同运行，也就是需要多个Data Thinker系统协同。为此我们扩展Data Thinker为CODA -- http://tab.d-thinker.org/showthread.php?tid=16288

当前计划是在wp289-{1,2}安装一焘think/sage, 在wp289-3安装一套，形成一个CODA系统。

在wp289-1, wp289-3各安装一套simpleserv访问点，分别服务上述两套Sage系统。

客户端可访问任何一个访问点进行查询。

RE: BIBO backend service - zhihao - 10-06-2020 12:28 PM

(10-06-2020 11:49 AM)lingu Wrote:
(09-29-2020 09:56 AM)lingu Wrote: Pls also create a new CT for the bibo backend.

Created?

Quote:a2. Add $tree to $codb in its admin CT. This usually is implemented by modifying a config file, such as gitolite.conf

I'm following instruction in Add a code tree to cod , a2 says I need modify gitolite.conf, So I request an account in http://tab.d-thinker.org/showthread.php?tid=7655&pid=123461#pid123461

RE: BIBO backend service - lingu - 10-06-2020 04:46 PM

(10-06-2020 11:51 AM)lingu Wrote: BIBO后台应支持高并发访问，并提供几乎不间断服务。

为此我们需要多个BIBO后台协同运行，也就是需要多个Data Thinker系统协同。为此我们扩展Data Thinker为CODA -- http://tab.d-thinker.org/showthread.php?tid=16288

当前计划是在wp289-{1,2}安装一焘think/sage, 在wp289-3安装一套，形成一个CODA系统。

在wp289-1, wp289-3各安装一套simpleserv访问点，分别服务上述两套Sage系统。

客户端可访问任何一个访问点进行查询。

zhihao pls review this design.

RE: BIBO backend service - lingu - 10-06-2020 04:51 PM

(10-06-2020 12:28 PM)zhihao Wrote:
(10-06-2020 11:49 AM)lingu Wrote:
(09-29-2020 09:56 AM)lingu Wrote: Pls also create a new CT for the bibo backend.

Created?

Quote:a2. Add $tree to $codb in its admin CT. This usually is implemented by modifying a config file, such as gitolite.conf
I'm following instruction in Add a code tree to cod , a2 says

Have you done a1?

Quote: I need modify gitolite.conf, So I request an account in http://tab.d-thinker.org/showthread.php?tid=7655&pid=123461#pid123461

gitolite.conf is in a CT itself. So you do not need access the VM.

After the new CT's info is ready, I will add the new CT's info to codb4, then you can continue to add files to the new CT.

RE: BIBO backend service - zhihao - 10-06-2020 05:11 PM

(10-06-2020 04:51 PM)lingu Wrote:
(10-06-2020 12:28 PM)zhihao Wrote:
(10-06-2020 11:49 AM)lingu Wrote:
(09-29-2020 09:56 AM)lingu Wrote: Pls also create a new CT for the bibo backend.

Created?

Quote:a2. Add $tree to $codb in its admin CT. This usually is implemented by modifying a config file, such as gitolite.conf
I'm following instruction in Add a code tree to cod , a2 says

Have you done a1?

yes, I add CTs to cod://cod/CodExt.py as follows:

Code:

"bibo": {"proto":"git",

        "repository": "bibo",

        "habitat": "bibo",

        "username": "git",

        "default_branch": "rc",

        "loc": "codb4.d-thinker.org"},

"bibo_backend": {"proto":"git",

        "repository": "bibo_backend",

        "habitat": "bibo",

        "username": "git",

        "default_branch": "rc",

        "loc": "codb4.d-thinker.org"},

RE: BIBO backend service - zhihao - 10-06-2020 05:19 PM

(10-06-2020 04:46 PM)lingu Wrote:
(10-06-2020 11:51 AM)lingu Wrote: BIBO后台应支持高并发访问，并提供几乎不间断服务。

为此我们需要多个BIBO后台协同运行，也就是需要多个Data Thinker系统协同。为此我们扩展Data Thinker为CODA -- http://tab.d-thinker.org/showthread.php?tid=16288

当前计划是在wp289-{1,2}安装一焘think/sage, 在wp289-3安装一套，形成一个CODA系统。

在wp289-1, wp289-3各安装一套simpleserv访问点，分别服务上述两套Sage系统。

客户端可访问任何一个访问点进行查询。

zhihao pls review this design.

good to me and updated headpost.

RE: BIBO backend service - lingu - 10-06-2020 07:26 PM

(10-06-2020 05:11 PM)zhihao Wrote: yes, I add CTs to cod://cod/CodExt.py as follows:

Code:

"bibo": {"proto":"git", "repository": "bibo", "habitat": "bibo", "username": "git", "default_branch": "rc", "loc": "codb4.d-thinker.org"}, "bibo_backend": {"proto":"git", "repository": "bibo_backend", "habitat": "bibo", "username": "git", "default_branch": "rc", "loc": "codb4.d-thinker.org"},

I changed bibo_backend to biboback, changed bibo to reside on codb3, and I changed their habitats to be prada.

Pls contact the guardians of codb3, such as Zijian, to add the CT bibo.

I have added CT biboback to codb4.

RE: BIBO backend service - zhihao - 10-08-2020 12:09 AM

(10-06-2020 07:26 PM)lingu Wrote:
(10-06-2020 05:11 PM)zhihao Wrote: yes, I add CTs to cod://cod/CodExt.py as follows:

Code:

"bibo": {"proto":"git", "repository": "bibo", "habitat": "bibo", "username": "git", "default_branch": "rc", "loc": "codb4.d-thinker.org"}, "bibo_backend": {"proto":"git", "repository": "bibo_backend", "habitat": "bibo", "username": "git", "default_branch": "rc", "loc": "codb4.d-thinker.org"},

I changed bibo_backend to biboback, changed bibo to reside on codb3, and I changed their habitats to be prada.

Pls contact the guardians of codb3, such as Zijian, to add the CT bibo.

I have added CT biboback to codb4.

added the CT bibo

RE: BIBO backend service - lingu - 11-30-2020 02:43 PM

(09-26-2020 03:54 PM)lingu Wrote: We should make BIBO backend a service.

An example of starting sequence for the backend is in http://rar.shufangkeji.com:60380/showthread.php?tid=9169&pid=43605#pid43605

Designers' info
BIBO后台应支持高并发访问，并提供几乎不间断服务。

为此我们需要多个BIBO后台协同运行，也就是需要多个Data Thinker系统协同。为此我们扩展Data Thinker为CODA -- http://tab.d-thinker.org/showthread.php?tid=16288

当前计划是在wp289-{1,2}安装一焘think/sage, 在wp289-3安装一套，形成一个CODA系统。

在wp289-1, wp289-3各安装一套simpleserv访问点，分别服务上述两套Sage系统。

客户端可访问任何一个访问点进行查询。

I think we can combine the D info with http://rar.shufangkeji.com:60380/showthread.php?tid=9162

I'll reorg the text and link tab9162 as D info.

RE: BIBO backend service - zhihao - 02-25-2021 06:26 PM

driver: 微信群内记录表示案例检索有问题

Quote:检察院呀！土豆:
@lingu 刚刚客户反应以前能检索到两个的案例现在有一个检索不出来了，大概是什么原因

lingu:
可能是有bug，但暂时没法下结论。先看看日志然后找合适的时间试试能否重现这个问题

建议将群内两个日志logs.zip和ynetlog.zip记录于GCP://org/troubleshoot/procuratorate/log/

RR lingu

RE: BIBO backend service - zhihao - 06-02-2021 12:22 PM

20210531 检察院系统无法运行，收集日志存放于GCP://org/procuratorate/supreme/log/wp289b/

RR lingu

I copy them to limbo305-1:/home/supporters/bibo_log too

RE: BIBO backend service - lingu - 06-02-2021 05:04 PM

(02-25-2021 06:26 PM)zhihao Wrote: driver: 微信群内记录表示案例检索有问题

Quote:检察院呀！土豆:
@lingu 刚刚客户反应以前能检索到两个的案例现在有一个检索不出来了，大概是什么原因

lingu:
可能是有bug，但暂时没法下结论。先看看日志然后找合适的时间试试能否重现这个问题

建议将群内两个日志logs.zip和ynetlog.zip记录于GCP://org/troubleshoot/procuratorate/log/

RR lingu

OK but is troubleshoot an org?
change the gcp to gcp://org/procuratorate/troubleshoot/log/xyz

xyz is a name indicating what this log is.

RE: BIBO backend service - lingu - 06-02-2021 05:30 PM

(06-02-2021 12:22 PM)zhihao Wrote: 20210531 检察院系统无法运行，收集日志存放于GCP://org/procuratorate/supreme/log/wp289b/

RR lingu

OK but pls change to GCP://org/procuratorate/supreme/lwp289b/log

Quote:I copy them to limbo305-1:/home/supporters/bibo_log too

Why do you copy there? If you plan to copy to a local dir, you'd better not define the gcp.

RE: BIBO backend service - lingu - 06-06-2021 12:08 AM

It looks like sage crashes after the walker on wp289b-6 runs 624 tasks.

To test, I am running wake_sage in tmux t3s on gm292-3.

Code:

[sage@limbo305-3 log]$ ~/sage/bin/wake_sage 

sage talks to Sage every minute.

You may read the depgrid info in the ATPD log.

...

RE: BIBO backend service - lingu - 07-01-2021 12:07 AM

20210630/Qi reports that startbibo could not start ncat. Zhihao helped them reinstall cod://bibo but it did not help. lingu went to the site and found wp286b-6 has a full root. After clearing bibo history log (about 20GB), we were able to start sage. There we some strange auntie tasks in sage, I ran sage_clear on wp289b-1 to clear old tasks. Then I killed some old processes using port 62818, and ran startbibo again. This time it works.