BIBO on limbo305

[lingu]

(08-13-2021 09:39 AM)lingu Wrote:  i re-trun startbibo, and the dir is created.

another problem

Code:

[sage@limbo305-1 test]$ make test cases=delete_zeng
Testing delete_zeng
    run test  ./test_delete_zeng.sh
    sending delete request /thinker/local/soft/bibo/app/test/processed/del_zengxingliang.json to localhost:62818
./test_delete_zeng.sh: line 22: /thinker/local/soft/bibo/app/test/processed/response-2021-08-13.txt: Permission denied
rm: remove write-protected regular empty file '/thinker/local/soft/bibo/app/test/processed/response-2021-08-13.txt'? ^Cmake: *** [Makefile:22: test] Interrupt

[sage@limbo305-1 test]$ ll /thinker/local/soft/bibo/app/test/processed/response-2021-08-13.txt
ls: cannot access '/thinker/local/soft/bibo/app/test/processed/response-2021-08-13.txt': No such file or directory
[sage@limbo305-1 test]$

but it doesnt happen any more.

[zhihao]

I can't find log of procone2.py for case like 27 in limbo305

Code:

[root@limbo305-1 ~]# grep -nir "opmode"  /thinker/bin/ephemeral/
[root@limbo305-1 ~]# grep -nir "opmode"  /thinker/local/today/users | tail
/thinker/local/today/users/sage/atpash.log-15:4:  opmode: dedicate
/thinker/local/today/users/sage/atpash.log-15:25:  opmode: dedicate
/thinker/local/today/users/sage/atpash.log-15:46:  opmode: dedicate
/thinker/local/today/users/sage/atpash.log-15:69:  opmode: dedicate
/thinker/local/today/users/sage/atpash.log-15:90:  opmode: dedicate
/thinker/local/today/users/sage/atpash.log-15:111:  opmode: dedicate
/thinker/local/today/users/sage/atpash.log-15:132:  opmode: dedicate
/thinker/local/today/users/sage/atpash.log-15:153:  opmode: dedicate
/thinker/local/today/users/sage/atpash.log-15:181:  opmode: dedicate
/thinker/local/today/users/root/procone2-19.log:3:  opmode: dedicate
[root@limbo305-1 ~]#
[root@limbo305-1 ~]# dexer 'ls -al /thinker/local/today/users/sage/| grep atpash'
10.36.1.49: ls -al /thinker/local/today/users/sage/| grep atpash
-rw-rw-rw-  1 sage sage    7959 Aug 13 11:19 atpash.log
-rwxrwxrwx  1 sage sage   10403 Aug 13 11:19 atpash.log-12
-rwxrwxrwx  1 sage sage   10403 Aug 13 11:19 atpash.log-15
-rwxrwxrwx  1 sage sage   10403 Aug 13 11:19 atpash.log-18
-rw-------  1 root root   12288 Aug 13 11:28 .atpash.log-21.swp
-rwxrwxrwx  1 sage sage    3351 Aug 13 11:19 atpash.log-24
-rwxrwxrwx  1 sage sage   18067 Aug 13 11:19 atpash.log-3
-rwxrwxrwx  1 sage sage   18067 Aug 13 11:19 atpash.log-6
-rwxrwxrwx  1 sage sage   15491 Aug 13 11:19 atpash.log-9

10.36.2.50: ls -al /thinker/local/today/users/sage/| grep atpash
-rw-rw-rw-  1 sage sage    6565 Aug 13 11:19 atpash.log
-rwxrwxrwx  1 sage sage    7637 Aug 13 11:19 atpash.log-10
-rwxrwxrwx  1 sage sage    7430 Aug 13 11:19 atpash.log-13
-rwxrwxrwx  1 sage sage    7430 Aug 13 11:19 atpash.log-16
-rwxrwxrwx  1 sage sage    8668 Aug 13 11:19 atpash.log-19
-rwxrwxrwx  1 sage sage    3979 Aug 13 11:19 atpash.log-25
-rwxrwxrwx  1 sage sage    7591 Aug 13 11:19 atpash.log-4
-rwxrwxrwx  1 sage sage    7591 Aug 13 11:19 atpash.log-7

10.36.3.51: ls -al /thinker/local/today/users/sage/| grep atpash
-rw-rw-rw-   1 sage sage     19862 Aug 13 11:19 atpash.log
-rwxrwxrwx   1 sage sage      4454 Aug 13 11:19 atpash.log-11
-rwxrwxrwx   1 sage sage      4454 Aug 13 11:19 atpash.log-14
-rwxrwxrwx   1 sage sage       513 May 21 18:51 atpash.log-15
-rwxrwxrwx   1 sage sage      4454 Aug 13 11:19 atpash.log-17
-rwxrwxrwx   1 sage sage      3128 May 22 19:24 atpash.log-18
-rwxrwxrwx   1 sage sage      3128 May 22 19:24 atpash.log-21
-rwxrwxrwx   1 sage sage      1488 Aug 13 11:19 atpash.log-26
-rwxrwxrwx   1 sage sage     25746 Aug  8 18:35 atpash.log-32
-rwxrwxrwx   1 sage sage     24491 Aug  8 18:35 atpash.log-35
-rwxrwxrwx   1 sage sage     21974 Aug  8 18:35 atpash.log-38
-rwxrwxrwx   1 sage sage     21974 Aug  8 18:35 atpash.log-41
-rwxrwxrwx   1 sage sage      7306 Aug  8 18:35 atpash.log-44
-rwxrwxrwx   1 sage sage      6264 Aug  8 18:35 atpash.log-47
-rwxrwxrwx   1 sage sage      7325 Aug 13 11:19 atpash.log-5
-rwxrwxrwx   1 sage sage      5222 Aug  8 18:35 atpash.log-50
-rwxrwxrwx   1 sage sage      4176 Aug  8 18:35 atpash.log-53
-rwxrwxrwx   1 sage sage      2084 Aug  8 18:35 atpash.log-56
-rwxrwxrwx   1 sage sage      2084 Aug  8 18:35 atpash.log-59
-rwxrwxrwx   1 sage sage      1042 Aug  8 18:35 atpash.log-62

[root@limbo305-1 ~]#

[lingu]

(08-13-2021 11:42 AM)zhihao Wrote:  find utilib error, because it can't find file "/thinker/etc/soft/sites/limbo305-tc/bibo.pcf"

Code:

[root@limbo305-2 ~]# su sage
[sage@limbo305-2 root]$ /thinker/local/soft/bibo/plug/procone2.py --qid 0 --tycano 19
Traceback (most recent call last):
  File "/thinker/local/soft/bibo/plug/procone2.py", line 23, in <module>
    rb, gmic = worksite.learnSiteMic(modname="bibo")
  File "/thinker/local/forest/util/utilib/worksite.py", line 88, in learnSiteMic
    print >> sys.stderr, "Unable to locate configuration file " + micpfn
NameError: global name 'micpfn' is not defined
[sage@limbo305-2 root]$

that's strange. i believe i installed bibo on limbo305-2.

the file maybe should be /thinker/etc/soft/sites/limbo305-tc/bibo/mic.pcf

but that error should not kill the process.

utilib is perhaps old on limbo305. i reinstalled utilib to update it on all 3 nodes.

Code:

[root@limbo305-2 root]# dexer ' /thinker/local/forest/util/utilib/installx utilib'
10.36.1.49:  /thinker/local/forest/util/utilib/installx utilib
/thinker/globe/udata/root/Limbo/Packages/limbo305-tc
Installing /thinker/globe/udata/root/Limbo/Packages/limbo305-tc/utilib.tar.gz ... success

10.36.2.50:  /thinker/local/forest/util/utilib/installx utilib
/thinker/globe/udata/root/Limbo/Packages/limbo305-tc
Installing /thinker/globe/udata/root/Limbo/Packages/limbo305-tc/utilib.tar.gz ... success

10.36.3.51:  /thinker/local/forest/util/utilib/installx utilib
/thinker/globe/udata/root/Limbo/Packages/limbo305-tc
Installing /thinker/globe/udata/root/Limbo/Packages/limbo305-tc/utilib.tar.gz ... success

[root@limbo305-2 root]#

thanks for spotting the problem, but pls work on wp289 so that we can make progress on both systems.

[zhihao]

update bibo for limbo305

Code:

[root@limbo305-1 ~]# dexer "/thinker/local/forest/util/utilib/installx bibo"
10.36.1.49: /thinker/local/forest/util/utilib/installx bibo
/thinker/globe/udata/root/Limbo/Packages/limbo305-tc
Installing /thinker/globe/udata/root/Limbo/Packages/limbo305-tc/bibo.tar.gz ... fail
  installx is unable to install bibo [0813-12:20:36]

10.36.2.50: /thinker/local/forest/util/utilib/installx bibo
/thinker/globe/udata/root/Limbo/Packages/limbo305-tc
Installing /thinker/globe/udata/root/Limbo/Packages/limbo305-tc/bibo.tar.gz ... fail
  installx is unable to install bibo [0813-12:20:58]

10.36.3.51: /thinker/local/forest/util/utilib/installx bibo
/thinker/globe/udata/root/Limbo/Packages/limbo305-tc
Installing /thinker/globe/udata/root/Limbo/Packages/limbo305-tc/bibo.tar.gz ... fail
  installx is unable to install bibo [0813-12:21:20]

[root@limbo305-1 ~]#

[lingu]

bibo seems to be flapping.

Code:

session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:03]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:03]
    session returns 0
  recycling at reqno 7
  simpleserve v2 for reqno 0 [0813-14:17:03]
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:07]

The reason was probably another simpleserv running. i killed all simpleserv and ncat then start bibo again. There is no more flapping.

zhihao - pls design to solve this issue. If the port is occupied, we'd better report the accurate error.

[lingu]

Strangely, atpa always termiante after creating 10.seen in task 10.

Code:

2021.08.13 14:59:52 (0f):   creating dir /thinker/fastdata/bibo/e/res2//0/10                                                          │·············
2021.08.13 14:59:52 (0f):     mkdir returns - ffffffffffffffff 0 0 0                                                                  │·············
2021.08.13 14:59:52 (0f):         atpa.runx kissed back  - 1 0 0 a                                                                    │·············
2021.08.13 14:59:52 (0f):       atpa.runx runs ..- /bin/sh /thinker/globe/.think/run/atpa.sh 0 10.. 0x0 0xa 0x1                       │·············
2021.08.13 14:59:54 (0f):       atpa.runx creating ..- /thinker/fastdata/bibo/e/res2//0/10.seen.. 0x0 0xa 0x1                         │·············

--- That's because of static logging not output to atpa.log-10

i am adding logging info to http://rar.shufangkeji.com:60380/showthr...p?tid=9227

[zhihao]

(08-13-2021 03:18 PM)lingu Wrote:  bibo seems to be flapping.

Code:

session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:03]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:03]
    session returns 0
  recycling at reqno 7
  simpleserve v2 for reqno 0 [0813-14:17:03]
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:06]
    session returns 0
      listen on port 62818 with /thinker/local/soft/bibo/plug/talk2thinker2.sh [0813-14:17:07]

The reason was probably another simpleserv running. i killed all simpleserv and ncat then start bibo again. There is no more flapping.

zhihao - pls design to solve this issue. If the port is occupied, we'd better report the accurate error.

OK, RR in BIBO (D)

[lingu]

(08-13-2021 12:33 PM)zhihao Wrote:  I can't find log of procone2.py for case like 27 in limbo305

i cannot find it, either. It should be in the normal log location /thinker/local/today/users/$USER

i am debugging it.

--- i find it is in atpash.log

Code:

[sage@limbo305-2 sage]$ /bin/sh /thinker/globe/.think/run/atpa.sh 32 10
blologing.....
using log_pfn from envar None
pfn /thinker/local/today/users/sage/atpash.log-10

So it is expected behavior. i will update the log info in rar9227

[lingu]

Testing on limbo305.

Code:

[sage@limbo305-1 test]$ make test cases=insert_zeng
Testing insert_zeng
    run test  ./test_insert_zeng.sh
sending insert request ./processed/zengxingliang-newkey.json
return code: 200
insert success
[sage@limbo305-1 test]$

Then I ran sage_restart on gm305-3, then test inquire_zeng.

Code:

[sage@limbo305-1 test]$ make test cases=inquire_zeng
Testing inquire_zeng
    run test  ./test_inquire_zeng.sh
sending inquire request ./processed/zengxingliang-inquire.json to localhost:62818
{
"code": 404,
"msg": "not found",
"children": [
]
}

[lingu]

It seems malloc() has an issue and the kissup function does not work in Filer.read()

When we read to ccc which is a local array, it is fine.

Code:

llRead = fread(ccc/*pDest*/, 1, Bize(), pfiMain_);

But pDesk, which is malloc'ed, hangs.

--- i am not able to pinpoint the problem. It is quite cryptic and i file a bug -- http://tab.d-thinker.org/showthread.php?...#pid134800

i work around the problem by adding a 1MB margin.