Threaded Mode | Linear Mode

***lingu*** · 09-23-2020, 01:05 PM

(09-23-2020 12:39 PM)zhihao Wrote:
(09-03-2020 11:53 PM)lingu Wrote: procone.py

Code:

在某一案例内容中搜索key{1,2,3}，构建"keys": []。

添加procone.py 伪代码，增加删除功能

main() {
set $tycajson_pfn as a insert.json
get $reqtype from $tycajson_pfn
if $reqtype is "delete"

Pls make both delete and query logic functins.

Quote: rename insert.json as insert2.json

You can use rid --insitu to do this. Otherwise, there may be complication when you have already a file insert2.json in the dir.

You should judge whether the file needs to be deleted. You can simply use re.search to search for the case ID and then judge whether the case ID matches the case ID to be deleted.

Quote: create file insert.json

How do you makes sure the new insert.json does not match any keywords?

Quote: else
generate query response
}
[/code]

should have RR'ed.

***lingu*** · (This post was last modified: 09-23-2020 01:28 PM by lingu.)

(09-23-2020 12:58 PM)zhihao Wrote:
(09-23-2020 12:14 PM)lingu Wrote:
Quote:tellThinker() {
set $casecnt as the number of cases
create a new directory $newdir, set its name as $casecnt + 1
copy $newcasepfn to $newdir

for each case; do
analyse cases with sage
done
}

This logic needs to handle multiple types of requests -- query, insert, delete. Pls design the switching logic and use different functions to do the work.

已加上switch逻辑

tellThinker() {
set $casecnt as the number of cases

switch $casetype {
case "insert":
create a new directory $newdir, set its name as $casecnt + 1
copy $newcasepfn to $newdir
generate insert success response

Make insert a function.

Quote: case "delete":
for each case; do
analyse cases with sage

That's too vague.
Again, make delete a function.

Quote: done
generate delete success response
case "query":
for each case; do
analyse cases with sage
done
generate query result response

make query a function too.

Quote: }
}
[/code]
RR lingu

Generally OK.

Quote:
(09-23-2020 12:14 PM)lingu Wrote: Also, design and implement the delete logic as we discussed -- for delete, rename the original file to be a different name, create a file in the original name and make sure it does not match any keywords.

Later, we need move the logic to a C++ program so that it can communicate with think in memory-to-memory data exchange.

删除逻辑设计在procone.py内，已在#49回复

Record a todo for he 'Later' part so that we can manage the task well.

09-23-2020, 01:33 PM

(09-23-2020 01:05 PM)lingu Wrote:
(09-23-2020 12:39 PM)zhihao Wrote:
(09-03-2020 11:53 PM)lingu Wrote: procone.py

Code:

在某一案例内容中搜索key{1,2,3}，构建"keys": []。

添加procone.py 伪代码，增加删除功能

main() {
set $tycajson_pfn as a insert.json
get $reqtype from $tycajson_pfn
if $reqtype is "delete"

Pls make both delete and query logic functins.

已修改并更新首文

Code:

main() {

    set $tycajson_pfn as a insert.json

    get $reqtype from $tycajson_pfn

    if $reqtype is "delete"

        delete()

    else

        query()

}

delete() {

    if $reqcaseid matches $caseid

        rename insert.json but don't cover existing file

    create file insert.json

}

query() {

    generate query response

}

(09-23-2020 01:05 PM)lingu Wrote:
Quote: rename insert.json as insert2.json

You can use rid --insitu to do this. Otherwise, there may be complication when you have already a file insert2.json in the dir.

You should judge whether the file needs to be deleted. You can simply use re.search to search for the case ID and then judge whether the case ID matches the case ID to be deleted.

已添加判断语句，判断caseid匹配后删除

Code:

delete() {

    if $reqcaseid matches $caseid

        rename insert.json but don't cover existing file

    create file insert.json

}

(09-23-2020 01:05 PM)lingu Wrote:
Quote: create file insert.json

How do you makes sure the new insert.json does not match any keywords?

这个我不理解什么意思，是要让procone.py可以获取key，但是值都要为空吗？

(09-23-2020 01:05 PM)lingu Wrote:
Quote: else
generate query response
}
[/code]

should have RR'ed.

已添加RR

09-23-2020, 01:58 PM

(09-23-2020 01:26 PM)lingu Wrote:
(09-23-2020 12:58 PM)zhihao Wrote:
(09-23-2020 12:14 PM)lingu Wrote:
Quote:tellThinker() {
set $casecnt as the number of cases
create a new directory $newdir, set its name as $casecnt + 1
copy $newcasepfn to $newdir

for each case; do
analyse cases with sage
done
}

This logic needs to handle multiple types of requests -- query, insert, delete. Pls design the switching logic and use different functions to do the work.

已加上switch逻辑

tellThinker() {
set $casecnt as the number of cases

switch $casetype {
case "insert":
create a new directory $newdir, set its name as $casecnt + 1
copy $newcasepfn to $newdir
generate insert success response

Make insert a function.

已设为函数

Code:

insert() {

    create a new directory $newdir, set its name as $casecnt + 1

    copy $newcasepfn to $newdir

    generate insert success response

}

(09-23-2020 01:26 PM)lingu Wrote:
Quote: case "delete":
for each case; do
analyse cases with sage

That's too vague.
Again, make delete a function.

已设为函数

Code:

delete() {

    for each case; do

        analyse cases with sage

    done

    generate delete success response

}

(09-23-2020 01:26 PM)lingu Wrote:
Quote: done
generate delete success response
case "query":
for each case; do
analyse cases with sage
done
generate query result response

make query a function too.

已设为函数

Code:

query() {

    for each case; do

        analyse cases with sage

    done

    generate query result response

}

(09-23-2020 01:26 PM)lingu Wrote:
Quote: }
}
[/code]
RR lingu

Generally OK.

已将修改后的如下内容更新首文

tellThinker() informs think about the request.

Code:

tellThinker() {

    set $casecnt as the number of cases

    switch $casetype {

        case "insert":

            insert()

        case "delete":

            delete()

        case "query":

            query()

    }

}

insert()

Code:

insert() {

    create a new directory $newdir, set its name as $casecnt + 1

    copy $newcasepfn to $newdir

    generate insert success response

}

delete()

Code:

delete() {

    for each case; do

        analyse cases with sage

    done

    generate delete success response

}

query()

Code:

query() {

    for each case; do

        analyse cases with sage

    done

    generate query result response

}

(09-23-2020 01:26 PM)lingu Wrote:
Quote:
(09-23-2020 12:14 PM)lingu Wrote: Also, design and implement the delete logic as we discussed -- for delete, rename the original file to be a different name, create a file in the original name and make sure it does not match any keywords.

Later, we need move the logic to a C++ program so that it can communicate with think in memory-to-memory data exchange.

删除逻辑设计在procone.py内，已在#49回复

Record a todo for he 'Later' part so that we can manage the task well.

add "[todo] implement delete function in procone.py" in headpost

***lingu*** · 09-23-2020, 03:34 PM

(09-23-2020 01:33 PM)zhihao Wrote:
Quote:How do you makes sure the new insert.json does not match any keywords?

这个我不理解什么意思，是要让procone.py可以获取key，但是值都要为空吗？

就是不要让procone.py以后在query的时候对已经被删除的文件达成任何匹配。

09-23-2020, 03:45 PM

(09-03-2020 11:53 PM)lingu Wrote: procone.py

delete()

Code:

delete() { if $reqcaseid matches $caseid rename insert.json but don't cover existing file create file insert.json }

首文delete() 中的 create file insert.json修改为 create empty file insert.json

09-23-2020, 05:07 PM

在procone.py和talk2thinker.sh中添加并push了删除语句

在 zhihao@g1b-2 screen simpleserv内开启simpleserv服务

Code:

[zhihao@g1b-2 procuratorate]$ ./simpleserv.sh

limbo30-1内提交del_zengxingliang.json请求，返回404错误

Code:

[zhihao@limbo30-1 processed]$ cat del_zengxingliang.json 

{"type":"delete","caseid":"67be261c9e6311eabceb005056c00001"}

[zhihao@limbo30-1 processed]$ 

[zhihao@limbo30-1 processed]$ ( cat del_zengxingliang.json; sleep 18 ) | ncat trajan.d-thinker.org 20050

{

"code": 404,

"msg": "not found",

"children": [

 ]

}

[zhihao@limbo30-1 processed]$ pwd

/data/20200906.1/processed

在zhihao@g1b-2 screen zh-scr内检查日志，看到talk2thinker.log日志和talk2thinker.sh的不符，检查后发现日志是从文件thinkres.py 154行开始记录。

Code:

[zhihao@g1b-2 zhihao]$ ll simpleserv.log talk2thinker-x.log 

-rw-rw-r-- 1 zhihao zhihao 264840 Sep 23 16:00 simpleserv.log

-rw-rw-r-- 1 zhihao zhihao  37376 Sep 23 16:00 talk2thinker-x.log

[zhihao@g1b-2 zhihao]$

simpleserv.log

Code:

working [0923-16:00:05] 

      received: /{"type":"delete","caseid":"67be261c9e6311eabceb005056c00001"}/ ..more 

      received: // ..more 

      received a blank line 

  talk2thinker reqno: 1 

    received new case in /thinker/storage/udata/lingu/testy/procuratorate/tests/newcase/newcase.txt [0923-16:00:06] 

      ts file /thinker/storage/udata/lingu/testy/procuratorate/tests/manytasks.ts generated 

      torun   sage_user=sage2 /thinker/globe/.think/run/auntie register /thinker/storage/udata/lingu/testy/procuratorate/tests/manytasks.ts 

      auntie completes with exit code 0 

    collecting results from /thinker/storage/udata/lingu/testy/procuratorate/tests/res2 

Sending results: [0923-16:00:08] 

{

"code": 404,

"msg": "not found",

"children": [

 ]

}

{

"code": 404,

"msg": "not found",

"children": [

 ]

}

      for exit status 1, we recycle thinker 

    session returns 0 

[zhihao@g1b-2 zhihao]$

talk2thinker.log

Code:

[2020-09-23 16:00:08] talk2thinker starts

  tycano: x

  more args 

talking to thinker

      scrutinize /thinker/storage/udata/lingu/testy/procuratorate/tests/res2

      scrutinize fns res_16.txt-tmp res_6.txt res.res res_8.txt res_4.txt res_17.txt-tmp res_7.txt res_1.txt res_5.txt res_12.txt res_10.txt res_13.txt res_3.txt res_2.txt res_9.txt res_11.txt res_15.txt res_18.txt-tmp res_19.txt-tmp res_14.txt .res.res.swp

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_6.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_8.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_4.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_7.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_1.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_5.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_12.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_10.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_13.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_3.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_2.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_9.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_11.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_15.txt

          to mv /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_14.txt

      juxtoposed with 10 cases

    producing output without search results

    thinkres completed with /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res.res

[zhihao@g1b-2 zhihao]$

***lingu*** · (This post was last modified: 09-23-2020 05:47 PM by lingu.)

(09-23-2020 05:07 PM)zhihao Wrote: 在zhihao@g1b-2 screen zh-scr内检查日志，看到talk2thinker.log日志和talk2thinker.sh的不符，检查后发现日志是从文件thinkres.py 154行开始记录。

Yes -- some messages are confusing. I have changed them in cod://prada commit e228632.

I think you should look into procone's log to see whether deletion took place.

I looked into one log file.

Code:

[2020-09-23 16:00:07] procone starts

  tycano: 16

  more args

  keys in /thinker/storage/udata/lingu/testy/procuratorate/tests/typical/16/insert.json

    keys ready

    tyca json loaded

    opening intermediate file /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_16.txt-tmp

    opened result file /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_16.txt-tmp

But I am not sure if it is deleting.

09-23-2020, 06:20 PM

(09-23-2020 05:43 PM)lingu Wrote:
(09-23-2020 05:07 PM)zhihao Wrote: 在zhihao@g1b-2 screen zh-scr内检查日志，看到talk2thinker.log日志和talk2thinker.sh的不符，检查后发现日志是从文件thinkres.py 154行开始记录。

Yes -- some messages are confusing. I have changed them in cod://prada commit e228632.

I think you should look into procone's log to see whether deletion took place.

I looked into one log file.

Code:

[2020-09-23 16:00:07] procone starts tycano: 16 more args keys in /thinker/storage/udata/lingu/testy/procuratorate/tests/typical/16/insert.json keys ready tyca json loaded opening intermediate file /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_16.txt-tmp opened result file /thinker/storage/udata/lingu/testy/procuratorate/tests/res2/res_16.txt-tmp

But I am not sure if it is deleting.

我生成的log中，procone.log最后修改日期都为20200919.

Code:

[zhihao@g1b-2 zhihao]$ pwd

/thinker/local/today/users/zhihao

[zhihao@g1b-2 zhihao]$ ll

total 600

-rw-rw-r-- 1 zhihao zhihao 114659 Sep 23 17:17 auntie.log

-rw-rw-r-- 1 zhihao zhihao   6878 Sep 22 17:53 coolauntie.log

-rw------- 1 zhihao zhihao     42 Sep 22 20:45 nohup.out

-rw-rw-r-- 1 zhihao zhihao   4228 Sep 20 18:01 procone-10.log

-rw-rw-r-- 1 zhihao zhihao   1260 Sep 19 15:26 procone-11.log

-rw-rw-r-- 1 zhihao zhihao    691 Sep 19 15:26 procone-12.log

-rw-rw-r-- 1 zhihao zhihao    691 Sep 19 15:27 procone-13.log

-rw-rw-r-- 1 zhihao zhihao    630 Sep 19 15:27 procone-14.log

-rw-rw-r-- 1 zhihao zhihao    691 Sep 19 15:27 procone-16.log

-rw-rw-r-- 1 zhihao zhihao    691 Sep 19 15:27 procone-17.log

-rw-rw-r-- 1 zhihao zhihao    691 Sep 19 15:27 procone-19.log

-rw-rw-r-- 1 zhihao zhihao   1250 Sep 20 18:06 procone-1.log

-rw-rw-r-- 1 zhihao zhihao    691 Sep 19 15:27 procone-23.log

-rw-rw-r-- 1 zhihao zhihao    630 Sep 19 15:27 procone-24.log

-rw-rw-r-- 1 zhihao zhihao    691 Sep 19 15:27 procone-25.log

-rw-rw-r-- 1 zhihao zhihao   1372 Sep 20 18:06 procone-2.log

-rw-rw-r-- 1 zhihao zhihao   2058 Sep 19 15:26 procone-3.log

-rw-rw-r-- 1 zhihao zhihao    686 Sep 19 15:26 procone-4.log

-rw-rw-r-- 1 zhihao zhihao    808 Sep 19 15:26 procone-5.log

-rw-rw-r-- 1 zhihao zhihao    686 Sep 19 15:26 procone-6.log

-rw-rw-r-- 1 zhihao zhihao    625 Sep 19 15:26 procone-7.log

-rw-rw-r-- 1 zhihao zhihao    686 Sep 19 15:26 procone-8.log

-rw-rw-r-- 1 zhihao zhihao    625 Sep 19 15:26 procone-9.log

-rw-rw-r-- 1 zhihao zhihao  66305 Sep 23 17:17 sage.log

-rw-rw-r-- 1 zhihao zhihao 266700 Sep 23 17:17 simpleserv.log

-rw-rw-r-- 1 zhihao zhihao  37376 Sep 23 16:00 talk2thinker-x.log

-rw-rw-r-- 1 zhihao zhihao   3922 Sep 23 17:17 thinkeres-x.log

[zhihao@g1b-2 zhihao]$

***lingu*** · (This post was last modified: 09-23-2020 06:28 PM by lingu.)

(09-23-2020 06:20 PM)zhihao Wrote:

Code:

[zhihao@g1b-2 zhihao]$ pwd /thinker/local/today/users/zhihao

need go to /thinker/local/today/users/sage2 because sage runs the individual tasks and sage runs using user sage2.

I suggest you split D info to anthoher thread and split the BIBO system on g1b as another thread. This is the overview thread for BIBO and we do not want it to be too crowded.