Rigorous and Reliable (RAR)
BIBO (D) [1 BUG, 2 TODOs] - Printable Version

+- Rigorous and Reliable (RAR) (http://rar.shufangkeji.com:60380)
+-- Forum: 大计算 (/forumdisplay.php?fid=257)
+--- Forum: 应用 (/forumdisplay.php?fid=363)
+---- Forum: BIBO (/forumdisplay.php?fid=602)
+---- Thread: BIBO (D) [1 BUG, 2 TODOs] (/showthread.php?tid=9162)

BIBO (D) [1 BUG, 2 TODOs] - zhihao - 09-24-2020 01:05 PM

[bug] delete_case function does not return 200
TODO1: improve search function, 13 seconds is not fast
TODO2 : add concurrent query support.

This is disigners' info for cod://prada/procuratorate

目前BIBO的运行比较复杂，要启动simpleserv和Data Thinker，并联通端口。
另外，建议请把BIBO变成一个服务，用systemctl start bibo.service启动。
BIBO backend service为BIBO系统的后台服务。

BIBO work environment 用于设置BIBO系统的运行环境

simpleserv.sh
Code:
监听62818端口，执行$bibo_base/talk2thinker.sh

talk2thinker.sh
Code:
work() {
    read $newcasepfn till meet blank line

    triage()
}

triage() informs think about the request.
Code:
$tspfn: sage tasks

triage() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert_case()
        case "delete":
            delete_case()
        case "query":
            query_case()
        case "select":
            select_case()
    }

    set $sage_user as sage2
    use program auntie  to register $tspfn
}

insert_case()
bibo maintains $casecnt as the total number of cases in the corpus.
Code:
insert_case() {
    create a new directory $newdir, set its name as ++$casecnt
    copy $newcasepfn to be $newdir/content
    auntie up --here=$newdir $casecnt
    generate insert success response
}

delete_case()
Code:
delete_case() {
    clear file $tspfn
    $reqcaseid = requested case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    auntie register tasks
    generate delete success response
}

query_case()
Code:
query_case() {
    clear file $tspfn
    for each case; do
        set option $tycano for procone.py
    done
    auntie register tasks
    generate query result response
}

select_case()
Code:
select_case() {
    clear file $tspfn
    $reqcaseid = requested case id
    for each case; do
      register tasks:
        procone.py --tycano=$tycano --reqtype=$reqtype --reqcaseid=$reqcaseid
    done
    getresults()
}

getResults() collect returned results from think.
Code:
getresults() {
        scrutinize() {
                generate response for each case
        }
        generate response for all cases
        return response
}

scrutinize() examines intermediate result files and generates an overall result file.

---
20220718/zhihao: add 2 todos
20210813/lingu: detail insert
20210810/lingu: link query case
20201010/zhihao: add BIBO work environment
20200929/zhihao: add BIBO backend service
20200926/zhihao: add select_case()
20200926/lingu: run $bibo_base/talk2thinker.sh
20200925/zhihao: add delete_case bug
20200925/zhihao: move procone.py
20200924/zhihao: update title to BIBO (D)
20200924/zhihao: update tellThinker function

RE: prada -- D - zhihao - 09-24-2020 02:08 PM

修改首文tellThinker函数及其内插入删除和查询函数的设计信息
(09-24-2020 01:05 PM)zhihao Wrote:  tellThinker() informs think about the request.
Code:
tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert()
        case "delete":
            delete()
        case "query":
            query()
    }
}
修改为
tellThinker() informs think about the request.
Code:
$tspfn: sage tasks

tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert_case()
        case "delete":
            delete_case()
        case "query":
            query_case()
    }

    change user to sage2
    use program auntie  to register $tspfn
}

(09-24-2020 01:05 PM)zhihao Wrote:  insert()
Code:
insert() {
    create a new directory $newdir, set its name as $casecnt + 1
    copy $newcasepfn to $newdir
    generate insert success response
}
修改为
insert_case()
Code:
insert_case() {
    create a new directory $newdir, set its name as $casecnt + 1
    copy $newcasepfn to $newdir
    generate insert success response
}

(09-24-2020 01:05 PM)zhihao Wrote:  delete()
Code:
delete() {
    for each case; do
        analyse cases with sage
    done
    generate delete success response
}
修改为
delete_case()
Code:
delete_case() {
    clear file $tspfn
    set $reqcaseid as request case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    generate delete success response
}

(09-24-2020 01:05 PM)zhihao Wrote:  query()
Code:
query() {
    for each case; do
        analyse cases with sage
    done
    generate query result response
}
修改为
query_case()
Code:
query_case() {
    clear file $tspfn
    for each case; do
        set option $tycano for procone.py
    done
    generate query result response
}

请审核上述tellThinker函数及插入、删除、查询函数设计信息
RR lingu

RE: prada -- D - lingu - 09-24-2020 02:32 PM

(09-24-2020 02:08 PM)zhihao Wrote:  修改首文tellThinker函数及其内插入删除和查询函数的设计信息
(09-24-2020 01:05 PM)zhihao Wrote:  tellThinker() informs think about the request.
Code:
tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert()
        case "delete":
            delete()
        case "query":
            query()
    }
}
修改为
tellThinker() informs think about the request.
Code:
$tspfn: sage tasks

tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert_case()
        case "delete":
            delete_case()
        case "query":
            query_case()
    }

    change user to sage2
    use program auntie  to register $tspfn
}

(09-24-2020 01:05 PM)zhihao Wrote:  insert()
Code:
insert() {
    create a new directory $newdir, set its name as $casecnt + 1
    copy $newcasepfn to $newdir
    generate insert success response
}
修改为
insert_case()
Code:
insert_case() {
    create a new directory $newdir, set its name as $casecnt + 1
    copy $newcasepfn to $newdir
    generate insert success response
}

(09-24-2020 01:05 PM)zhihao Wrote:  delete()
Code:
delete() {
    for each case; do
        analyse cases with sage
    done
    generate delete success response
}
修改为
delete_case()
Code:
delete_case() {
    clear file $tspfn
    set $reqcaseid as request case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    generate delete success response
}

(09-24-2020 01:05 PM)zhihao Wrote:  query()
Code:
query() {
    for each case; do
        analyse cases with sage
    done
    generate query result response
}
修改为
query_case()
Code:
query_case() {
    clear file $tspfn
    for each case; do
        set option $tycano for procone.py
    done
    generate query result response
}

请审核上述tellThinker函数及插入、删除、查询函数设计信息
RR lingu

That's too much to review. You should have proposed the changes one by one -- one for insert, one for delete, etc.

I think it is fine as the first version. Pls update to the headpost then we improve further.

Pls name this thread BIBO (D). Prada is different.

Pls mv this to the Prada board -- http://rar.shufangkeji.com:60380/forumdisplay.php?fid=406

RE: prada -- D - zhihao - 09-24-2020 03:48 PM

(09-24-2020 02:32 PM)lingu Wrote:  That's too much to review. You should have proposed the changes one by one -- one for insert, one for delete, etc.

I think it is fine as the first version. Pls update to the headpost then we improve further.

Pls name this thread BIBO (D). Prada is different.

Pls mv this to the Prada board -- http://rar.shufangkeji.com:60380/forumdisplay.php?fid=406

moved

RE: BIBO (D) - zhihao - 09-24-2020 06:01 PM

在cod://prada内添加了Makefile逻辑，添加maketst.sh。目前make test可测试插入删除。

如下为在screen zh-scr内执行的结果
Code:
[zhihao@g1b-2 prada]$ make test
casecnt before insert: 52
inserting /data/20200906.1/processed/zengxingliang.json
{
"code": 200,
"msg": "ok",
"children": []
}
insert finished

inserted file :/thinker/globe/udata/lingu/testy/procuratorate/tests/typical/53/insert.json
{"type":"insert","query":"","caseid":"","insert":{"keys":[{"key1":"破坏计算机信息系统","key2":"智能手机终端","key3":"计算机信息系统","detailid":"087b2085e4a911ea987a005056c00001"},{"key1":"破坏计算机信息系统","key2":"智能手机终端","key3":"破坏计算机信息系统","detailid":"087b5ca4e4a911ea987a005056c00001"},{"key1":"破坏计算机信息系统","key2":"智能手机终端","key3":"牵连犯","detailid":"087b61a1e4a911ea987a005056c00001"}],"caseid":"67be261c9e6311eabceb005056c00001","casetitle":"曾兴亮、王玉生破坏计算机信息系统案","caserate":"5.00","casetype":"最高人民检察院指导性案例","caseaction":"破坏计算机信息系统罪","caseyear":2017,"content":"第九批指导性案例(曾兴亮、王玉生破坏计算机信息系统案) （检例第35号） 【关键词】 破坏计算机信息系统 智能手机终端 远程锁定 【基本案情】 被告人曾兴亮，男，1997年8月生，农民。 被告人王玉生，男，1992年2月生，农民。 2016年10月至11月，被告人曾兴亮与王玉生结伙或者单独使用聊天社交软件，冒充年轻女性与被害人聊天，谎称自己的苹果手机因故障无法登录“iCloud”（云存储）​，请被害人代为登录，诱骗被害人先注销其苹果手机上原有的ID，再使用被告人提供的ID及密码登录。随后，曾、王二人立即在电脑上使用新的ID及密码登录苹果官方网站，利​用苹果手机相关功能将被害人的手机设置修改，并使用“密码保护问题”修改该ID的密码，从而远程锁定被害人的苹果手机。曾、王二人再在其个人电脑上，用网络聊天软件与被害​人联系，以解锁为条件索要钱财。采用这种方式，曾兴亮单独或合伙作案共21起，涉及苹果手机22部，锁定苹果手机21部，索得人民币合计7290元；王玉生参与作案12起​，涉及苹果手机12部，锁定苹果手机11部，索得人民币合计4750元。2016年11月24日，二人被公安机关抓获。 【诉讼过程和结果】 本案由江苏省海安县人民检察院于2016年12月23日以被告人曾兴亮、王玉生犯破坏计算机信息系统罪向海安县人民法院提起公诉。2017年1月20日，海安县人民法院作​出判决，认定被告人曾兴亮、王玉生的行为构成破坏计算机信息系统罪，分别判处有期徒刑一年三个月、有期徒刑六个月。一审宣判后，二被告人未上诉，判决已生效。 【要　旨】 智能手机终端，应当认定为刑法保护的计算机信息系统。锁定智能手机导致不能使用的行为，可认定为破坏计算机信息系统。 【指导意义】 计算机信息系统包括计算机、网络设备、通信设备、自动化控制设备等。智能手机和计算机一样，使用独立的操作系统、独立的运行空间，可以由用户自行安装软件等程序，并可以通​过移动通讯网络实现无线网络接入，应当认定为刑法上的“计算机信息系统”。 行为人通过修改被害人手机的登录密码，远程锁定被害人的智能手机设备，使之成为无法开机的“僵尸机”，属于对计算机信息系统功能进行修改、干扰的行为。造成10台以上智能​手机系统不能正常运行，符合刑法第二百八十六条破坏计算机信息系统罪构成要件中“对计算机信息系统功能进行修改、干扰”“后果严重”的情形，构成破坏计算机信息系统罪。 行为人采用非法手段锁定手机后以解锁为条件，索要钱财，在数额较大或多次敲诈的情况下，其目的行为又构成敲诈勒索罪。在这类犯罪案件中，手段行为构成的破坏计算机信息系统​罪与目的行为构成的敲诈勒索罪之间成立牵连犯。牵连犯应当从一重罪处断。破坏计算机信息系统罪后果严重的情况下，法定刑为五年以下有期徒刑或者拘役；敲诈勒索罪在数额较大​的情况下，法定刑为三年以下有期徒刑、拘役或者管制，并处或者单处罚金。本案应以重罪即破坏计算机信息系统罪论处。 【相关法律规定】 《中华人民共和国刑法》 第二百八十六条 违反国家规定，对计算机信息系统功能进行删除、修改、增加、干扰，造成计算机信息系统不能正常运行，后果严重的，处五年以下有期徒刑或者拘役；后果特别严重的，处五年以上​有期徒刑。 第二百七十四条 敲诈勒索公私财物，数额较大或者多次敲诈勒索的，处三年以下有期徒刑、拘役或者管制，并处或者单处罚金；数额巨大或者有其他严重情节的，处三年以上十年以下有期徒刑，并处​罚金；数额特别巨大或者有其他特别严重情节的，处十年以上有期徒刑，并处罚金。 《最高人民法院、最高人民检察院关于办理危害计算机信息系统安全刑事案件应用法律若干问题的解释》 第十一条 本解释所称“计算机信息系统”和“计算机系统”，是指具备自动处理数据功能的系统，包括计算机、网络设备、通信设备、自动化控制设备等。 …… 《最高人民法院、最高人民检察院关于办理敲诈勒索刑事案件适用法律若干问题的解释》 第一条 敲诈勒索公私财物价值二千元至五千元以上、三万元至十万元以上、三十万元至五十万元以上的，应当分别认定为刑法第二百七十四条规定的“数额较大”、“数额巨大”、“数额特​别巨大”。 各省、自治区、直辖市高级人民法院、人民检察院可以根据本地区经济发展状况和社会治安状况，在前款规定的数额幅度 内，共同研究确定本地区执行的具体数额标准，报最高人民法院、最高人民检察院批准。 《江苏省高级人民法院、江苏省人民检察院、江苏省公安厅关于我省执行敲诈勒索公私财物“数额较大”、“数额巨大”、“数额特别巨大”标准的意见》 根据《最高人民法院、最高人民检察院关于办理敲诈勒索刑事案件适用法律若干问题的解释》的规定，结合我省经济发展和社会治安实际状况，确定我省执行刑法第二百七十四条规定​的敲诈勒索公私财物“数额较大”、“数额巨大”、“数额特别巨大”标准如下： 一、敲诈勒索公私财物价值人民币四千元以上的，为“数额较大”； 二、敲诈勒索公私财物价值人民币六万元以上的，为“数额巨大”； ……"}}


casecnt after insert: 53

deleting /data/20200906.1/processed/del_zengxingliang.json
{
"code": 200,
"msg": "ok",
"children": []
}
delete finished

file content after deleted
{"type": "","query": "","caseid": "","insert": {"keys": [] }}

[zhihao@g1b-2 prada]$


RE: BIBO (D) - lingu - 09-24-2020 07:22 PM

(09-24-2020 06:01 PM)zhihao Wrote:  在cod://prada内添加了Makefile逻辑，添加maketst.sh。目前make test可测试插入删除。

should move it to dir procuratorate. bibo is not prada.

where is the info about the tests?


Quote:如下为在screen zh-scr内执行的结果
Code:
[zhihao@g1b-2 prada]$ make test
...
file content after deleted
{"type": "","query": "","caseid": "","insert": {"keys": [] }}

[zhihao@g1b-2 prada]$

It asks me for your password.
Code:
18:20:32 lingu@g1b-2:/home/lingu/forest/prada/prada> make test
casecnt before insert: 58
inserting /data/20200906.1/processed/zengxingliang.json
zhihao@10.6.64.21's password:


RE: BIBO (D) - zhihao - 09-24-2020 07:57 PM

(09-24-2020 07:22 PM)lingu Wrote:  
(09-24-2020 06:01 PM)zhihao Wrote:  在cod://prada内添加了Makefile逻辑，添加maketst.sh。目前make test可测试插入删除。

should move it to dir procuratorate. bibo is not prada.

where is the info about the tests?
在cod://prada/procuratorate/ 内创建Makefile和maketest.sh。
maketest.sh D info:
Code:
show case count before insert
send insert request to trajan.d-thinker.org 20050

output inserted case file
show case count after insert

send delete request to trajan.d-thinker.org 20050

RR lingu

(09-24-2020 07:22 PM)lingu Wrote:  
Quote:如下为在screen zh-scr内执行的结果
Code:
[zhihao@g1b-2 prada]$ make test
...
file content after deleted
{"type": "","query": "","caseid": "","insert": {"keys": [] }}

[zhihao@g1b-2 prada]$

It asks me for your password.
Code:
18:20:32 lingu@g1b-2:/home/lingu/forest/prada/prada> make test
casecnt before insert: 58
inserting /data/20200906.1/processed/zengxingliang.json
zhihao@10.6.64.21's password:

之前获取插入和删除测试案例时是登陆limbo30-1获取的，所以需要密码，目前已将测试案例存放在$bibo_base/processed
screen zh-scr现在可以执行make test测试

RE: BIBO (D) - lingu - 09-24-2020 08:19 PM

(09-24-2020 01:05 PM)zhihao Wrote:  talk2thinker.sh
Code:
work() {
    read $newcasepfn till meet blank line

    tellThinker()
    getresults()
}

getresults() may have trouble for insert and delete because they do not generate "results".

Quote:tellThinker() informs think about the request.
Code:
$tspfn: sage tasks

tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert_case()
        case "delete":
            delete_case()
        case "query":
            query_case()
    }

    change user to sage2

It does not "change user". The program just specify sage_user=sage2, right?

Quote: use program auntie to register $tspfn
}

We should not register tasks for insert. So I think we need move this 'auntie register' to the functions for query and delete.
--- 20200924/lingu moved the auntie register to functions delete_case() and query_case()

RE: BIBO (D) - lingu - 09-24-2020 08:22 PM

(09-24-2020 07:57 PM)zhihao Wrote:  
(09-24-2020 07:22 PM)lingu Wrote:  
(09-24-2020 06:01 PM)zhihao Wrote:  在cod://prada内添加了Makefile逻辑，添加maketst.sh。目前make test可测试插入删除。

should move it to dir procuratorate. bibo is not prada.

where is the info about the tests?
在cod://prada/procuratorate/ 内创建Makefile和maketest.sh。
Dont make a big test of all tests. Make small tests and combine them.

I think you can rename maketest.sh to be test_crds.sh -- it creates, reads, deletes and selects a case. The create, read, delete and select ops can be individual tests in their own scripts. The crds test invokes them in sequence.

Quote:maketest.sh D info:
Code:
show case count before insert
send insert request to trajan.d-thinker.org 20050

output inserted case file
show case count after insert

send delete request to trajan.d-thinker.org 20050

RR lingu

OK as the first version. Make sure to ALWAYS RECORD INFO when designing.

No need to specify D info for tests. Just write OMUD together. Create a thread for that.

Quote:
(09-24-2020 07:22 PM)lingu Wrote:  It asks me for your password.
Code:
18:20:32 lingu@g1b-2:/home/lingu/forest/prada/prada> make test
casecnt before insert: 58
inserting /data/20200906.1/processed/zengxingliang.json
zhihao@10.6.64.21's password:

之前获取插入和删除测试案例时是登陆limbo30-1获取的，所以需要密码，目前已将测试案例存放在$bibo_base/processed
screen zh-scr现在可以执行make test测试

测试应该用各自用户进行，或规定用户。screen是用来协同调试用的。注意分清各种情况。

RE: prada -- D - lingu - 09-24-2020 08:26 PM

(09-24-2020 02:08 PM)zhihao Wrote:  tellThinker() informs think about the request.
Code:
tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert()
        case "delete":
            delete()
        case "query":
            query()
    }
}

There is a 'select' function.

Pls design the 'select' function. You can ask collaborators if you are not sure what 'select' is expected to do.

Pls also add a query test and a select test.

RE: BIBO (D) - lingu - 09-24-2020 10:28 PM

(09-24-2020 01:05 PM)zhihao Wrote:  talk2thinker.sh

This is not the best name. But let's tolerate it.

Quote:
Code:
work() {
    read $newcasepfn till meet blank line

    tellThinker()
    getresults()
}

tellThinker() informs think about the request.
Code:
$tspfn: sage tasks

tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert_case()
        case "delete":
            delete_case()
        case "query":
            query_case()
    }

    change user to sage2
    use program auntie  to register $tspfn
}

tellThinker should really talk to think. But insert() does not. So the name is inadequate. I suggest we rename it to triage() because it differentiates different types of requests and dispatch them to different functions.

RR zhihao

RE: BIBO (D) - lingu - 09-24-2020 10:32 PM

(09-24-2020 01:05 PM)zhihao Wrote:  delete_case()
Code:
delete_case() {
    clear file $tspfn
    set $reqcaseid as request case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    generate delete success response
}

This is problematic because we give a result '200' before Sage invokes think to run the tasks. It is possible something goes wrong with the delete, but the client receives '200' and think everything is fine. That's a bug.

I am removing the success response.

Then the delete_case function does not return 200, and it is a problem.
Code:
21:40:44 lingu@g1b-2:/home/lingu/forest/prada/prada> make -C procuratorate test
make: Entering directory '/home/lingu/forest/prada/prada/procuratorate'
casecnt before insert: 72
inserting /thinker/storage/soft/bibo/processed/zengxingliang.json
{
"code": 200,
"msg": "ok",
"children": []
}
insert finished

inserted file :/thinker/globe/udata/lingu/testy/procuratorate/tests/typical/73/insert.json
{"type":"insert","query":"","caseid":"","insert":{"keys":[{"key1":"破▒计▒▒信▒系▒","key


casecnt after insert: 73

deleting /thinker/storage/soft/bibo/processed/del_zengxingliang.json
delete finished

file content after deleted
{"type": "","query": "","caseid": "","insert": {"keys": [] }}

make: Leaving directory '/home/lingu/forest/prada/prada/procuratorate'
21:41:06 lingu@g1b-2:/home/lingu/forest/prada/prada>


RE: BIBO (D) - lingu - 09-24-2020 10:52 PM

(09-24-2020 01:05 PM)zhihao Wrote:  procone.py
Code:
在某一案例内容中搜索key{1,2,3}，构建"keys": []。
Code:
main() {
    set $tycajson_pfn as a insert.json
    get $reqtype from $tycajson_pfn
    if $reqtype is "delete"
        delete()

This delete() is different from delete_case() so the name is fine. But I still feel it is too "general" a name. How about we rename it deleteIfMatch?

Quote: else
query()
}
[/code]

procone is quite complex. Pls separate it as a standalone thread.

RE: BIBO (D) - lingu - 09-24-2020 10:55 PM

(09-24-2020 01:05 PM)zhihao Wrote:  delete()
Code:
delete() {

I see the implementation is not a function, yet. Pls make it a function soon.

Quote: if $reqcaseid matches $caseid
rename insert.json but don't cover existing file

Just say 'rid --insitu insert.json'

Quote: create empty file insert.json
}
[/code]
It is not a really empty file, is it?

After the match-and-delete, we should generate a result file to report the result of the deletion. If there is no match and no deletion, the result file is a nil file -- a file with zero bytes.

RE: BIBO (D) - zhihao - 09-24-2020 11:58 PM

(09-24-2020 10:55 PM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  delete()
Code:
delete() {

I see the implementation is not a function, yet. Pls make it a function soon.

我这里看procone.py是已经将删除功能设为函数，且今天没有修改过procone.py，也许是我没有push该文件
Code:
...
glog = None
reqcaseid = None

def delete(caseid, tycano, reqcaseid):
    global toneroot
    tycajson_pfn = toneroot + "/typical/" + tycano + "/insert.json"
    glog.flog("    delete() attempts to match and delete " + str(req
    glog.flog("      tycajson_pfn " + tycajson_pfn)
    glog.flog("      caseid :" + str(caseid) + "reqcaseid: " + str(r
    if caseid == reqcaseid:
      glog.flog("      caseid == reqcaseid")
      cmdline.runOrExit("rid --insitu " + tycajson_pfn )
      fw = open(tycajson_pfn,"w")
      fw.write("{\"type\": \"\",\"query\": \"\",\"caseid\": \"___\",
{\"keys\": [] }} \n")
      fw.close()

def main():
  allops = sys.argv[1:]
...


RE: BIBO (D) - zhihao - 09-25-2020 12:18 AM

(09-24-2020 08:19 PM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  talk2thinker.sh
Code:
work() {
    read $newcasepfn till meet blank line

    tellThinker()
    getresults()
}

getresults() may have trouble for insert and delete because they do not generate "results".
将getresults()删除，后面再将其加到query_case()后执行。已更新首文为如下内容：
Code:
work() {
    read $newcasepfn till meet blank line

    tellThinker()
}

(09-24-2020 08:19 PM)lingu Wrote:  
Quote:tellThinker() informs think about the request.
Code:
$tspfn: sage tasks

tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert_case()
        case "delete":
            delete_case()
        case "query":
            query_case()
    }

    change user to sage2

It does not "change user". The program just specify sage_user=sage2, right?
首文修改为
(( change user to sage2 -> set $sage_user as sage2)）

(09-24-2020 08:19 PM)lingu Wrote:  
Quote: use program auntie to register $tspfn
}

We should not register tasks for insert. So I think we need move this 'auntie register' to the functions for query and delete.
--- 20200924/lingu moved the auntie register to functions delete_case() and query_case()

OK, I'll reply headpost and update these two D info in #22

RE: BIBO (D) - zhihao - 09-25-2020 12:33 AM

(09-24-2020 08:22 PM)lingu Wrote:  
(09-24-2020 07:57 PM)zhihao Wrote:  
(09-24-2020 07:22 PM)lingu Wrote:  
(09-24-2020 06:01 PM)zhihao Wrote:  在cod://prada内添加了Makefile逻辑，添加maketst.sh。目前make test可测试插入删除。

should move it to dir procuratorate. bibo is not prada.

where is the info about the tests?
在cod://prada/procuratorate/ 内创建Makefile和maketest.sh。
Dont make a big test of all tests. Make small tests and combine them.

I think you can rename maketest.sh to be test_crds.sh -- it creates, reads, deletes and selects a case. The create, read, delete and select ops can be individual tests in their own scripts. The crds test invokes them in sequence.
renamed maketest.sh to test_crds.sh and created file "test_delete.sh test_insert.sh test_read.sh test_select.sh" in cod://prada/procuratorate

(09-24-2020 08:22 PM)lingu Wrote:  
Quote:maketest.sh D info:
Code:
show case count before insert
send insert request to trajan.d-thinker.org 20050

output inserted case file
show case count after insert

send delete request to trajan.d-thinker.org 20050

RR lingu

OK as the first version. Make sure to ALWAYS RECORD INFO when designing.

No need to specify D info for tests. Just write OMUD together. Create a thread for that.
created in test_crds.sh -- OMUD

RE: prada -- D - zhihao - 09-25-2020 12:51 AM

(09-24-2020 08:26 PM)lingu Wrote:  
(09-24-2020 02:08 PM)zhihao Wrote:  tellThinker() informs think about the request.
Code:
tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert()
        case "delete":
            delete()
        case "query":
            query()
    }
}

There is a 'select' function.
added a 'select' function to tellThinker()

(09-24-2020 08:26 PM)lingu Wrote:  Pls design the 'select' function. You can ask collaborators if you are not sure what 'select' is expected to do.

select function designed as follows:
Code:
select_case() {
    clear file $tspfn
    set $reqcaseid as request case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    getresults()
}
RR lingu

(09-24-2020 08:26 PM)lingu Wrote:  Pls also add a query test and a select test.

OK

RE: BIBO (D) - zhihao - 09-25-2020 12:53 AM

(09-24-2020 10:28 PM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  talk2thinker.sh

This is not the best name. But let's tolerate it.

Quote:
Code:
work() {
    read $newcasepfn till meet blank line

    tellThinker()
    getresults()
}

tellThinker() informs think about the request.
Code:
$tspfn: sage tasks

tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert_case()
        case "delete":
            delete_case()
        case "query":
            query_case()
    }

    change user to sage2
    use program auntie  to register $tspfn
}

tellThinker should really talk to think. But insert() does not. So the name is inadequate. I suggest we rename it to triage() because it differentiates different types of requests and dispatch them to different functions.

RR zhihao
LGTM and updated headpost and cod://prada/procuratorate/talk2thinker.sh

RE: BIBO (D) - zhihao - 09-25-2020 01:16 AM

(09-24-2020 10:52 PM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  procone.py
Code:
在某一案例内容中搜索key{1,2,3}，构建"keys": []。
Code:
main() {
    set $tycajson_pfn as a insert.json
    get $reqtype from $tycajson_pfn
    if $reqtype is "delete"
        delete()

This delete() is different from delete_case() so the name is fine. But I still feel it is too "general" a name. How about we rename it deleteIfMatch?
OK, updated headpost and cod.

(09-24-2020 10:52 PM)lingu Wrote:  
Quote: else
query()
}
[/code]

procone is quite complex. Pls separate it as a standalone thread.
created in procone.py -- OMUD

RE: BIBO (D) - zhihao - 09-25-2020 01:32 AM

(09-24-2020 10:55 PM)lingu Wrote:  
Quote: if $reqcaseid matches $caseid
rename insert.json but don't cover existing file

Just say 'rid --insitu insert.json'
updated headpost in procone.py -- OMUD

(09-24-2020 10:55 PM)lingu Wrote:  
Quote: create empty file insert.json
}
[/code]
It is not a really empty file, is it?

After the match-and-delete, we should generate a result file to report the result of the deletion. If there is no match and no deletion, the result file is a nil file -- a file with zero bytes.
updated in procone.py -- OMUD and added result file design
(( create empty file insert.json -> create file insert.json with request template which has empty value))

RE: BIBO (D) - zhihao - 09-25-2020 01:41 AM

(09-24-2020 01:05 PM)zhihao Wrote:  This is disigners' info for cod://prada/procuratorate

delete_case()
Code:
delete_case() {
    clear file $tspfn
    set $reqcaseid as request case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    generate delete success response
}

query_case()
Code:
query_case() {
    clear file $tspfn
    for each case; do
        set option $tycano for procone.py
    done
    generate query result response
}

add "auntie register tasks" for these two function to headpost
(( done -> done
auntie register tasks))

RE: BIBO (D) - zhihao - 09-25-2020 01:54 AM

(09-24-2020 10:32 PM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  delete_case()
Code:
delete_case() {
    clear file $tspfn
    set $reqcaseid as request case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    generate delete success response
}

This is problematic because we give a result '200' before Sage invokes think to run the tasks. It is possible something goes wrong with the delete, but the client receives '200' and think everything is fine. That's a bug.

I am removing the success response.

Then the delete_case function does not return 200, and it is a problem.
Code:
21:40:44 lingu@g1b-2:/home/lingu/forest/prada/prada> make -C procuratorate test
make: Entering directory '/home/lingu/forest/prada/prada/procuratorate'
casecnt before insert: 72
inserting /thinker/storage/soft/bibo/processed/zengxingliang.json
{
"code": 200,
"msg": "ok",
"children": []
}
insert finished

inserted file :/thinker/globe/udata/lingu/testy/procuratorate/tests/typical/73/insert.json
{"type":"insert","query":"","caseid":"","insert":{"keys":[{"key1":"破▒计▒▒信▒系▒","key


casecnt after insert: 73

deleting /thinker/storage/soft/bibo/processed/del_zengxingliang.json
delete finished

file content after deleted
{"type": "","query": "","caseid": "","insert": {"keys": [] }}

make: Leaving directory '/home/lingu/forest/prada/prada/procuratorate'
21:41:06 lingu@g1b-2:/home/lingu/forest/prada/prada>

recorded bug to headpost

RE: BIBO (D) - zhihao - 09-25-2020 02:27 PM

正在实现procone.py select功能，调试后发现cmdline命令似乎没有执行。
根据cmdline Ui写了个测试，无输出
Code:
[zhihao@g1b-2 20200914]$ cat test2.py
import sys

sys.path.append("/thinker/local/forest/util/utilib")
import cmdline

#cmdline.runOrExit("cat /thinker/storage/udata/lingu/testy/procuratorate/tests/typical/84/insert.json")
cmdline.runandread("ls")
[zhihao@g1b-2 20200914]$ python test2.py
[zhihao@g1b-2 20200914]$


RE: prada -- D - lingu - 09-26-2020 12:24 PM

(09-25-2020 12:51 AM)zhihao Wrote:  
(09-24-2020 08:26 PM)lingu Wrote:  
(09-24-2020 02:08 PM)zhihao Wrote:  tellThinker() informs think about the request.
Code:
tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert()
        case "delete":
            delete()
        case "query":
            query()
    }
}

There is a 'select' function.
added a 'select' function to tellThinker()

(09-24-2020 08:26 PM)lingu Wrote:  Pls design the 'select' function. You can ask collaborators if you are not sure what 'select' is expected to do.

select function designed as follows:

select_case() {
clear file $tspfn

OK.

Quote: set $reqcaseid as request case id

Pls change to
Code:
$reqcaseid = requested case id

Quote: for each case; do
set option $tycano, $reqtype and $reqcaseid for procone.py
done

Pls change to
Code:
register tasks:
        procone.py --tycano=$tycano --reqtype=$reqtype --reqcaseid=$reqcaseid

Quote: done

getresults()
}
[/code]
RR lingu

OK.

RE: BIBO (D) - lingu - 09-26-2020 12:34 PM

(09-25-2020 02:27 PM)zhihao Wrote:  正在实现procone.py select功能，调试后发现cmdline命令似乎没有执行。
根据cmdline Ui写了个测试，无输出
Code:
[zhihao@g1b-2 20200914]$ cat test2.py
import sys

sys.path.append("/thinker/local/forest/util/utilib")
import cmdline

#cmdline.runOrExit("cat /thinker/storage/udata/lingu/testy/procuratorate/tests/typical/84/insert.json")
cmdline.runandread("ls")
[zhihao@g1b-2 20200914]$ python test2.py
[zhihao@g1b-2 20200914]$

runOrExit does not show stdout and stderr. You can use runread() if you want to get the output. I am documenting some run command functions -- http://tab.d-thinker.org/showthread.php?tid=16267

RE: BIBO (D) - lingu - 09-26-2020 12:35 PM

(09-25-2020 01:54 AM)zhihao Wrote:  
(09-24-2020 10:32 PM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  delete_case()
Code:
delete_case() {
    clear file $tspfn
    set $reqcaseid as request case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    generate delete success response
}

This is problematic because we give a result '200' before Sage invokes think to run the tasks. It is possible something goes wrong with the delete, but the client receives '200' and think everything is fine. That's a bug.

I am removing the success response.

Then the delete_case function does not return 200, and it is a problem.
Code:
21:40:44 lingu@g1b-2:/home/lingu/forest/prada/prada> make -C procuratorate test
make: Entering directory '/home/lingu/forest/prada/prada/procuratorate'
casecnt before insert: 72
inserting /thinker/storage/soft/bibo/processed/zengxingliang.json
{
"code": 200,
"msg": "ok",
"children": []
}
insert finished

inserted file :/thinker/globe/udata/lingu/testy/procuratorate/tests/typical/73/insert.json
{"type":"insert","query":"","caseid":"","insert":{"keys":[{"key1":"破▒计▒▒信▒系▒","key


casecnt after insert: 73

deleting /thinker/storage/soft/bibo/processed/del_zengxingliang.json
delete finished

file content after deleted
{"type": "","query": "","caseid": "","insert": {"keys": [] }}

make: Leaving directory '/home/lingu/forest/prada/prada/procuratorate'
21:41:06 lingu@g1b-2:/home/lingu/forest/prada/prada>

recorded bug to headpost

I believe I have fixed this problem.

RE: BIBO (D) - lingu - 09-26-2020 01:49 PM

(09-25-2020 12:33 AM)zhihao Wrote:  
(09-24-2020 08:22 PM)lingu Wrote:  
(09-24-2020 07:57 PM)zhihao Wrote:  
(09-24-2020 07:22 PM)lingu Wrote:  
(09-24-2020 06:01 PM)zhihao Wrote:  在cod://prada内添加了Makefile逻辑，添加maketst.sh。目前make test可测试插入删除。

should move it to dir procuratorate. bibo is not prada.

where is the info about the tests?
在cod://prada/procuratorate/ 内创建Makefile和maketest.sh。
Dont make a big test of all tests. Make small tests and combine them.

I think you can rename maketest.sh to be test_crds.sh -- it creates, reads, deletes and selects a case. The create, read, delete and select ops can be individual tests in their own scripts. The crds test invokes them in sequence.
renamed maketest.sh to test_crds.sh and created file "test_delete.sh test_insert.sh test_read.sh test_select.sh" in cod://prada/procuratorate

(09-24-2020 08:22 PM)lingu Wrote:  
Quote:maketest.sh D info:
Code:
show case count before insert
send insert request to trajan.d-thinker.org 20050

output inserted case file
show case count after insert

send delete request to trajan.d-thinker.org 20050

RR lingu

OK as the first version. Make sure to ALWAYS RECORD INFO when designing.

No need to specify D info for tests. Just write OMUD together. Create a thread for that.
created in test_crds.sh -- OMUD

As I said " it creates, reads, deletes and selects a case. The create, read, delete and select ops can be individual tests in their own scripts. The crds test invokes them in sequence." There should be at least 4 programs -- programs for insert, read, delete and crd.

I am creating thread about bibo tests -- http://rar.shufangkeji.com:60380/showthread.php?tid=9167 Pls improve further.

RE: BIBO (D) - lingu - 09-26-2020 03:34 PM

(09-24-2020 01:05 PM)zhihao Wrote:  simpleserv.sh
Code:
监听62818端口，执行talk2thinker.sh

simpleserv still runs the talk2thinker.sh program in ~zhihao This confused me for 2 hours. You should have changed it to run programs in $bibo_base

I am changing it and modifying the headpost.

RE: BIBO (D) - lingu - 09-26-2020 03:45 PM

(09-24-2020 01:05 PM)zhihao Wrote:  getResults() collect returned results from think.
Code:
getresults() {
        scrutinize() {
                generate response for each case
        }
        generate response for all cases
        return response
}

scrutinize() is a bit complex and I am making it a standalone thread -- http://rar.shufangkeji.com:60380/showthread.php?tid=9168

RE: BIBO (D) - lingu - 09-26-2020 04:10 PM

(09-24-2020 01:05 PM)zhihao Wrote:  talk2thinker.sh
Code:
work() {
    read $newcasepfn till meet blank line

    triage()
}

triage() informs think about the request.
Code:
$tspfn: sage tasks

triage() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert_case()
        case "delete":
            delete_case()
        case "query":
            query_case()
        case "select":
            select_case()
    }

    set $sage_user as sage2
    use program auntie  to register $tspfn
}

triage and work should handle return status $? well. Otherwise the service would not work well. The return status handling has been messed up in the re-factoring. I am fixing them.

RE: prada -- D - zhihao - 09-26-2020 05:07 PM

(09-26-2020 12:24 PM)lingu Wrote:  
(09-25-2020 12:51 AM)zhihao Wrote:  
(09-24-2020 08:26 PM)lingu Wrote:  
(09-24-2020 02:08 PM)zhihao Wrote:  tellThinker() informs think about the request.
Code:
tellThinker() {
    set $casecnt as the number of cases

    switch $casetype {
        case "insert":
            insert()
        case "delete":
            delete()
        case "query":
            query()
    }
}

There is a 'select' function.
added a 'select' function to tellThinker()

(09-24-2020 08:26 PM)lingu Wrote:  Pls design the 'select' function. You can ask collaborators if you are not sure what 'select' is expected to do.

select function designed as follows:

select_case() {
clear file $tspfn

OK.

Quote: set $reqcaseid as request case id

Pls change to
Code:
$reqcaseid = requested case id

Quote: for each case; do
set option $tycano, $reqtype and $reqcaseid for procone.py
done

Pls change to
Code:
register tasks:
        procone.py --tycano=$tycano --reqtype=$reqtype --reqcaseid=$reqcaseid

Quote: done

getresults()
}
[/code]
RR lingu

OK.

updated as follows:
Code:
select_case() {
    clear file $tspfn
    $reqcaseid = requested case id
    for each case; do
      register tasks:
        procone.py --tycano=$tycano --reqtype=$reqtype --reqcaseid=$reqcaseid
    done
    getresults()
}


RE: BIBO (D) - lingu - 09-26-2020 05:11 PM

I tried select with a caseid without a match, but it still returns 200.
Code:
16:09:45 lingu@g1b-2:/thinker/globe/soft/bibo/processed> ( cat sel_no_match.json; sleep 10 ) | ncat trajan.d-thinker.org 20050
{
"code": 200,
"msg": "ok",
}

16:10:08 lingu@g1b-2:/thinker/globe/soft/bibo/processed> cat sel_no_match.json
{"type":"select","query":"","caseid":"67be261c9e6311eabceb005056","insert":{}}

16:10:14 lingu@g1b-2:/thinker/globe/soft/bibo/processed>


RE: BIBO (D) - lingu - 09-26-2020 05:18 PM

select_case does not work.

zhihao -5s lingu -2s

Note that -5s is a serious warning and may result in salary reduction.

RE: BIBO (D) - lingu - 09-26-2020 06:23 PM

i've made select_case work.

zhihao - pls design and implement a test for select, and add it to test_crds

RE: BIBO (D) - zhihao - 09-26-2020 06:30 PM

插入的返回值内不能有children，已将其删除。

目前查询功能返回404。试了第三次，可以正常查询了
Code:
[zhihao@limbo30-1 processed]$ ( cat chendengchang-with-blank-line.json;sleep 18 ) | ncat trajan.d-thinker.org 20050
{
"code": 404,
"msg": "not found",
"children": [
]
}

[zhihao@limbo30-1 processed]$

200 OK 和 404 error的返回值格式有一点问题，最后的的key后面应该没有逗号。
202009261747/zhihao: 已修复该问题

Code:
[zhihao@limbo30-1 processed]$ ( cat del_zengxingliang.json;sleep 18 ) | ncat trajan.d
-thinker.org 20050
{
"code": 404,
"msg": "not found",                                                                  }

[zhihao@limbo30-1 processed]$ ( cat zengxingliang.json;sleep 18 ) | ncat trajan.d-thinker.org 20050
{
"code": 200,
"msg": "ok",
}


RE: BIBO (D) - lingu - 09-26-2020 06:40 PM

(09-26-2020 06:30 PM)zhihao Wrote:  插入的返回值内不能有children，已将其删除。

目前查询功能返回404。试了第三次，可以正常查询了

[zhihao@limbo30-1 processed]$ ( cat chendengchang-with-blank-line.json;sleep 18 ) | ncat trajan.d-thinker.org 20050

dont use that any more. use 'make test'.

RE: BIBO (D) - zhihao - 09-29-2020 03:29 PM

(09-24-2020 01:05 PM)zhihao Wrote:  目前BIBO的运行比较复杂，要启动simpleserv和Data Thinker，并联通端口。
另外，建议请把BIBO变成一个服务，用systemctl start bibo.service启动。
首文添加BIBO backend service：
BIBO backend service为BIBO系统的后台服务。

RE: BIBO (D) - lingu - 10-09-2020 10:21 PM

(09-24-2020 01:05 PM)zhihao Wrote:  BIBO系统目前部署在$bibo_base: /thinker/globe/soft/bibo

志豪 - 抱歉我错误认为bibo_base是你改了值，但其实我们原来设计的就是在/thinker/globe/soft/bibo，此处我忘记了。

现在我们将bibo_base改为/thinker/local/soft/bibo 另外bibo_share是/thinker/globe/soft/bibo 请创建一个讨论 BIBO work environment 列出这些变量。

RE: BIBO (D) - zhihao - 10-10-2020 11:27 AM

(10-09-2020 10:21 PM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  BIBO系统目前部署在$bibo_base: /thinker/globe/soft/bibo

志豪 - 抱歉我错误认为bibo_base是你改了值，但其实我们原来设计的就是在/thinker/globe/soft/bibo，此处我忘记了。

现在我们将bibo_base改为/thinker/local/soft/bibo 另外bibo_share是/thinker/globe/soft/bibo 请创建一个讨论 BIBO work environment 列出这些变量。

已创建BIBO work environment

RE: BIBO (D) - zhihao - 10-10-2020 11:30 AM

(09-24-2020 01:05 PM)zhihao Wrote:  BIBO系统目前部署在$bibo_base: /thinker/globe/soft/bibo

首文修改为
((BIBO系统目前部署在$bibo_base: /thinker/globe/soft/bibo
->
BIBO work environment 用于设置BIBO系统的运行环境
))

RE: BIBO (D) - lingu - 10-12-2020 01:36 AM

(09-24-2020 01:05 PM)zhihao Wrote:  talk2thinker.sh

work() {
read $newcasepfn till meet blank line

Here backslash escape leads to trouble -- http://rar.shufangkeji.com:60380/showthread.php?tid=9167&pid=43883#pid43883

We should read the input without backslash escape interpretation -- 'read -r' instead of 'read'.

RR zhihao

--- working proactively, i've changed it in cod://bibo commit 6276848

RE: BIBO (D) - lingu - 11-30-2020 03:16 PM

(09-24-2020 01:05 PM)zhihao Wrote:  select_case()
select_case() {
clear file $tspfn
$reqcaseid = requested case id
for each case; do

case is not defined. for $reqcaseid ?

do we support multiple case ids?
Quote: register tasks:

register tasks is too slow -- it is a file interface to sage. We are should sacli to send the request through a socket interface to Sage. I'll send out a review request.

RE: BIBO (D) - lingu - 11-30-2020 03:22 PM

(09-24-2020 01:05 PM)zhihao Wrote:  
Code:
select_case()
[code]
select_case() {
    clear file $tspfn
    $reqcaseid = requested case id
    for each case; do
      register tasks:
        procone.py --tycano=$tycano --reqtype=$reqtype --reqcaseid=$reqcaseid
}
driver: task registration is too slow in wp289. We also need handle multiple concurrent queries so we should know a query ID (qid).

Proposal: I suggest we change the above three lines to
Code:
sacli --helper $bibo_helper match $newcasepfn
  if successful {
    qid=$(cat /tmp/bibo.qid)
  }

RR zhihao

RE: BIBO (D) - zhihao - 11-30-2020 05:03 PM

(11-30-2020 03:22 PM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  
Code:
select_case()
[code]
select_case() {
    clear file $tspfn
    $reqcaseid = requested case id
    for each case; do
      register tasks:
        procone.py --tycano=$tycano --reqtype=$reqtype --reqcaseid=$reqcaseid
}
driver: task registration is too slow in wp289. We also need handle multiple concurrent queries so we should know a query ID (qid).

Proposal: I suggest we change the above three lines to
Code:
sacli --helper $bibo_helper match $newcasepfn
  if successful {
    qid=$(cat /tmp/bibo.qid)
  }

RR zhihao

where is sacli's D info? I don't know how to use it and can't find it. So I can't tell which 3 lines should be replaced.

RE: BIBO (D) - zhihao - 05-22-2021 11:22 AM

update headopst:
BIBO use port 22,80,62818

RE: BIBO (D) - lingu - 05-22-2021 12:57 PM

(09-24-2020 01:05 PM)zhihao Wrote:  insert_case()
Code:
insert_case() {
    create a new directory $newdir, set its name as $casecnt + 1

should be $casecnt++

$newdir is $bibo_share/procuratorate/tests

I suggest we rename it $bibo_share/procuratorate/cases and use a convar bibo_case_dir to refer to it.

RR zhihao

Quote: copy $newcasepfn to $newdir

The file becomes $newdir/content

Quote:delete_case()
Code:
delete_case() {
    clear file $tspfn
    $reqcaseid = requested case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    auntie register tasks
    generate delete success response
}

We should also handle $bibo_case_dir/content and I suggest we just use rid --insitu to rename it.

RR zhihao

RE: BIBO (D) - lingu - 05-22-2021 01:01 PM

(10-12-2020 01:36 AM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  talk2thinker.sh

work() {
read $newcasepfn till meet blank line

Here backslash escape leads to trouble -- http://rar.shufangkeji.com:60380/showthread.php?tid=9167&pid=43883#pid43883

We should read the input without backslash escape interpretation -- 'read -r' instead of 'read'.

RR zhihao

--- working proactively, i've changed it in cod://bibo commit 6276848

?

RE: BIBO (D) - lingu - 05-22-2021 01:26 PM

(11-30-2020 05:03 PM)zhihao Wrote:  
(11-30-2020 03:22 PM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  
Code:
select_case()
[code]
select_case() {
    clear file $tspfn
    $reqcaseid = requested case id
    for each case; do
      register tasks:
        procone.py --tycano=$tycano --reqtype=$reqtype --reqcaseid=$reqcaseid
}
driver: task registration is too slow in wp289. We also need handle multiple concurrent queries so we should know a query ID (qid).

Proposal: I suggest we change the above three lines to
Code:
sacli --helper $bibo_helper match $newcasepfn
  if successful {
    qid=$(cat /tmp/bibo.qid)
  }

RR zhihao

where is sacli's D info? I don't know how to use it and can't find it. So I can't tell which 3 lines should be replaced.

"The above three lines" means the lines
-----
for each case; do
register tasks:
procone.py --tycano=$tycano --reqtype=$reqtype --reqcaseid=$reqcaseid
-----

I am sorry that sacli has little info. I think it evolved from sage-client. But we only have a bit info about sage-client like http://tab.d-thinker.org/showthread.php?tid=9730&pid=58644#pid58644 That is bad info recording. I am creating a thread about sacli -- http://tab.d-thinker.org/showthread.php?tid=16959

RE: BIBO (D) - lingu - 05-22-2021 01:28 PM

(05-22-2021 11:22 AM)zhihao Wrote:  update headopst:
BIBO use port 22,80,62818

How to make this info easy to find and use?

RE: BIBO (D) - zhihao - 05-24-2021 03:44 PM

(10-12-2020 01:36 AM)lingu Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  talk2thinker.sh

work() {
read $newcasepfn till meet blank line

Here backslash escape leads to trouble -- http://rar.shufangkeji.com:60380/showthread.php?tid=9167&pid=43883#pid43883

We should read the input without backslash escape interpretation -- 'read -r' instead of 'read'.

RR zhihao

--- working proactively, i've changed it in cod://bibo commit 6276848

good to me

RE: BIBO (D) - lingu - 06-03-2021 06:32 PM

We run sage_restart with a cron job with root on wp289-6. The cron job manifests as /etc/cron.d/cronsage owned by root with permission 644.

Code:
40 07 * * * sage source /home/sage/.bashrc && /home/sage/sage/bin/sage_restart

This is a system cron in user sage, and it seems the shell does not source .bashrc when running the job. Therefore, we need source .bashrc explicitly.

RE: BIBO (D) - lingu - 06-03-2021 06:33 PM

(05-22-2021 01:28 PM)lingu Wrote:  
(05-22-2021 11:22 AM)zhihao Wrote:  update headopst:
BIBO use port 22,80,62818

How to make this info easy to find and use?

?

RE: BIBO (D) - zhihao - 08-10-2021 01:42 PM

(09-24-2020 01:05 PM)zhihao Wrote:  delete_case()
Code:
delete_case() {
    clear file $tspfn
    $reqcaseid = requested case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    auntie register tasks
    generate delete success response
}

currently delete is limit to 90 cases, and it is not in D info, suggest delete it in code tree

RR lingu

Code:
function delete_case() {
  flog "  deleting case"
  truncate -s0 $tspfn
  reqcaseid=`jq '.caseid' $newcasepfn`
  tonicnt=$casecnt
  if (( $tonicnt > 90 )); then
    tonicnt=90
  fi


RE: BIBO (D) - zhihao - 08-10-2021 08:14 PM

lingu says it will call procone.py in the process of querying cases, but there seems only sacli is called. I can't find the call.

RE: BIBO (D) - lingu - 08-10-2021 08:23 PM

(09-24-2020 01:05 PM)zhihao Wrote:  query_case()
Code:
query_case() {
    clear file $tspfn
    for each case; do
        set option $tycano for procone.py
    done
    auntie register tasks
    generate query result response
}

i think we should move this out to another thread for clarity.

The current workflow is recorded in http://rar.shufangkeji.com:60380/showthread.php?tid=9227

RE: BIBO (D) - zhihao - 08-12-2021 10:28 AM

bibo generate blank json in commit 215052698dbe45a0e18a7a4ac4feac23aa21a7bf
Code:
-rw-rw-rw- 1 sage sage 156 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_126.txt.seen
-rw-rw-rw- 1 sage sage   0 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_127.txt.seen
-rw-rw-rw- 1 sage sage   0 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_128.txt.seen
-rw-rw-rw- 1 sage sage   0 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_129.txt.seen
-rw-rw-rw- 1 sage sage 156 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_12.txt.seen
-rw-rw-rw- 1 sage sage   0 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_130.txt.seen
-rw-rw-rw- 1 sage sage   0 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_131.txt.seen
-rw-rw-rw- 1 sage sage 156 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_132.txt.seen
-rw-rw-rw- 1 sage sage   0 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_133.txt.seen
-rw-rw-rw- 1 sage sage   0 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_134.txt.seen
-rw-rw-rw- 1 sage sage   0 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_135.txt.seen
-rw-rw-rw- 1 sage sage   0 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_136.txt.seen
-rw-rw-rw- 1 sage sage   0 Aug 12 00:52 /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_137.txt.seen
[root@wp289b-1 test]# less /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_132.txt.seen
  {
   "keys": [

   ],
   "caseid": "___",
   "caseyear": ,
   "casetitle": "",
   "casetype": "",
   "caserate": "",
   "caseaction": "",
   "score":
  }
[root@wp289b-1 test]# less /thinker/net/ynet/soft/bibo/procuratorate/cases/resprev/res_132.txt.seen


RE: BIBO (D) - zhihao - 08-12-2021 11:13 AM

(09-24-2020 01:05 PM)zhihao Wrote:  delete_case()
Code:
delete_case() {
    clear file $tspfn
    $reqcaseid = requested case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    auntie register tasks
    generate delete success response
}

delete logic above and update to below to make it can handle more than 90 cases
Code:
newcasepfn: content of new delete request
SACLI: sacli program
bibo_helper: bibo helper node ip

$SACLI --helper $bibo_helper match $newcasepfn

RR lingu

RE: BIBO (D) - zhihao - 08-12-2021 02:44 PM

(06-03-2021 06:33 PM)lingu Wrote:  
(05-22-2021 01:28 PM)lingu Wrote:  
(05-22-2021 11:22 AM)zhihao Wrote:  update headopst:
BIBO use port 22,80,62818

How to make this info easy to find and use?

?

record in U info is better.

RE: BIBO (D) - lingu - 08-13-2021 09:02 AM

(08-12-2021 11:13 AM)zhihao Wrote:  
(09-24-2020 01:05 PM)zhihao Wrote:  delete_case()
Code:
delete_case() {
    clear file $tspfn
    $reqcaseid = requested case id
    for each case; do
        set option $tycano, $reqtype and $reqcaseid for procone.py
    done
    auntie register tasks
    generate delete success response
}

delete logic above and update to below to make it can handle more than 90 cases
Code:
newcasepfn: content of new delete request
SACLI: sacli program
bibo_helper: bibo helper node ip

$SACLI --helper $bibo_helper match $newcasepfn

RR lingu

OK

RE: BIBO (D) - lingu - 08-13-2021 09:03 AM

(08-12-2021 02:44 PM)zhihao Wrote:  
(06-03-2021 06:33 PM)lingu Wrote:  
(05-22-2021 01:28 PM)lingu Wrote:  
(05-22-2021 11:22 AM)zhihao Wrote:  update headopst:
BIBO use port 22,80,62818

How to make this info easy to find and use?

?

record in U info is better.

Such port use info should be in M info. Do users need to know those ports?

RE: BIBO (D) - zhihao - 08-13-2021 11:06 AM

(08-13-2021 09:03 AM)lingu Wrote:  
(08-12-2021 02:44 PM)zhihao Wrote:  
(06-03-2021 06:33 PM)lingu Wrote:  
(05-22-2021 01:28 PM)lingu Wrote:  
(05-22-2021 11:22 AM)zhihao Wrote:  update headopst:
BIBO use port 22,80,62818

How to make this info easy to find and use?

?

record in U info is better.

Such port use info should be in M info. Do users need to know those ports?

No, engineers need

RE: BIBO (D) - zhihao - 08-13-2021 05:04 PM

(09-24-2020 01:05 PM)zhihao Wrote:  simpleserv.sh
Code:
监听62818端口，执行$bibo_base/talk2thinker.sh

update logic to
---
reqno: request number
bibo_server_port: port used by bibo
Code:
while true {                                          
  if ($reqno==0){                                    
    sage_clear                                        
  }                                                  
                                                      
  lsof -i :$bibo_server_port
  rr= $?                                        
                                                      
  if ($rr == 0) // port being used                    
    echo "port $bibo_server_port is already in use, exit"
    exit 9                                            
  }                                                  
                                                      
  listen port $bibo_server_port                      
  rr = run talk2thinker.sh                            
                                                      
  if ($rr != 0){                                      
    reqno=7                                          
  }                                                  
  if ( $reqno > 6 ){                                  
    flog "  recycling at reqno $reqno"                
    reqno=0                                          
  }                                                  
  else {                                  
    reqno=$reqno+1                                                                                    
  }                                                
}
---
RR lingu

RE: BIBO (D) - lingu - 08-14-2021 12:24 AM

(09-24-2020 01:05 PM)zhihao Wrote:  insert_case()
Code:
insert_case() {
    create a new directory $newdir, set its name as $casecnt + 1
    copy $newcasepfn to $newdir

It's copied as a file named content.

We also increment casecnt

After that, we run 'auntie up --here=$newdir $casecnt

Quote: generate insert success response
}
[/code]

i'll update the headpost.

RE: BIBO (D) - zhihao - 07-18-2022 12:41 PM

driver: http://rar.shufangkeji.com:60380/showthread.php?tid=9627&pid=54714#pid54714
---
add two todos
TODO1: improve search function, 13 seconds is not fast
TODO2 : add concurrent query support.