(07-12-2016 05:54 PM)YU_Xinjie Wrote:  This thread is the design of service management of Sage

The usage of service management is in
http://tab.d-thinker.org/showthread.php?tid=6164
http://tab.d-thinker.org/showthread.php?tid=6197

Format PPM

./format.sh

Code:

dt format -n NVPC k

Load metadata

./load.sh

Code:

set $cfgfile, $nodefile,  search file prefix($prefix).
echo 'Greppy system is starting.....'
tr '\n' from the $nodefile

function get_basic_info:
    get vpcs,space,ip,slot,filename, nf(how many parts one line divided by space) info from $nodefile, all vars are array var.

function make_link:
parameter: $file need to search
    if [ ! -f $file on $ip ] :
          echo '$file not on $ip" and exit
    else:
          make links with $file into  thinker stdin dir.

function gain_config() :
printf $space, $filesize, $dejavusize into $cfgfile

totalline = len of $nodefile

function gen_config()
mv vpc.$num and nrcs.$num to thinker runtime dir.

for i in range(0,totalline):
    get_basic_info()
    $host[i] = $ip
    if [ $nf -eq 2 ]  ##means file need to search
      make_link
      gain_config
   else
      print $space, 0, 0, 'n' to $cfgfile
      
gen_config()
call make datacfg to run size prog
call make read to run read prog

Start service

./start.sh

Code:

trap ctrl_c

if type ctrl_c; then
    echo "You type ctrl_c to terminate program"
    kill DT
    clean all object files and exit

if the 1st parameter  == 'nohup'; then
    nohup do_start &
else
    do_start

function do_start() {
    make run
}

Stop service

./stop.sh

Code:

dt slay -n NVPC k

Show status

./status.h

Code:

dexer --ips=$ips "ps aux" | grep -E 'vpc|nrc|scheduler|mem_home|atpd|auntie|ncat' | grep 'sage' | grep -v 'dexer' | grep -v 'grep'

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  The brief idea is to use auntie to send a file to glad_portal. Then check the file content.

Quote:1.
require sage_user can password-less login sage_user@sage_portal
copy private key of sage_user@sage_portal into $gb/conf/sage_key

Then glad operative user & sage_user can use sage_key to execute remote command to sage_user@sage_portal.

(09-07-2016 03:53 PM)lingu Wrote:  It seems to be quite complex. Do we need to really pay that complexity? If so, please let me know and I'll review the design in detail.

(09-07-2016 03:55 PM)lingu Wrote:  We still need to handle race condition for running multiple sage thinkers. But I think thinkers already have such conflict reporting capability. We just need to track the return status of start.sh and report an error as well as stop glad when errors occur.

If start.sh does not report the return status correctly, we need to perhaps improve start.sh.

(09-07-2016 03:58 PM)lingu Wrote:  

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  The brief idea is to use auntie to send a file to glad_portal. Then check the file content.

Directly running auntie without argumets can already check if sage is present.

(09-07-2016 04:11 PM)YU_Xinjie Wrote:  

(09-07-2016 03:53 PM)lingu Wrote:  It seems to be quite complex. Do we need to really pay that complexity? If so, please let me know and I'll review the design in detail.

The code I post is actually the script I develop for shbio. It works well for more than one week.
It is complex but robust.

I hope you can check it, so that I can put it into our glad/sage code tree.
You do not need to check every options of every command, because they work. But you'd better check the brief method. In brief, they are just a remote cmd wrapper for the start.sh/stop.sh/status.sh, except the function wait_sage_start, which uses auntie to check sage.

(09-07-2016 05:06 PM)lingu Wrote:  I will review. But this is TERRIBLE practice. Please make sure to get endorsement from another engineer before starting to implement an important function. If you ask me to review but I don't respond, please remind me for an important review.

If you need to spend half a day to implement some code, you should be prudent in writing the code directly. Instead, invite reviews. We may find a way to solve the problem in 2 hours.

If there is no response after reminders, you may decide to go ahead with implementing an important piece of code.

If it takes only 30min to implement something, you can go ahead and implement it without endorsement if you don't want to wait. If the change is insignificant, you may also skip the review.

It is generally not practical to define what is "important" or "significant". We rely on engineers' judgement.

This one -- how to start/stop sage and make it work with glad -- is certainly an important piece of design.

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  goal

1. the script can be used in both sage_portal & glad_portal to start/stop/status sage.
2. user can start sage before glad, and stop sage after glad.

Quote:The script would replace the current start.sh/stop.sh/status.sh in the future.
The script may inspire us about how to set up regression tests for sage.

Quote:[quote]
For shbio's project, I have already written a script /thinker/dstore/run/sage to support a part of the goal. I would try to generalize it.

config design

The brief idea is to use auntie to send a file to glad_portal. Then check the file content.

Quote:1.
require sage_user can password-less login sage_user@sage_portal

Quote:copy private key of sage_user@sage_portal into $gb/conf/sage_key

Quote:Then glad operative user & sage_user can use sage_key to execute remote command to sage_user@sage_portal.

2. The glad_portal IP can be get from $gb/conf/config.sh

Quote:3.
create a config file $gb/conf/sage_portal_ip to record the IP of sage_portal. Or we can also write it into $gb/conf/config.sh.

Quote:4. let sage_user@sage_portal can password-less login glad_user@glad_portal.

Quote:start sage

Code:

remote_cmd () {
ssh -q -o StrictHostKeyChecking=no -i $gb/conf/sage_key $sage_user@$sage_portal "$1"
}
[/quote]

Don't have to specify this -- everybody can find a way to run a remote program without being tremendously wrong. So it's not a key design detail.

We require engineers use psudocode to write down the design if it involves some logic. But code is not appropriate psudocode.

[quote]
wait_sage_start () {
content="This is test content."
echo $content > $curdir/${USER}_sage_testfile
test_file=/thinker/bin/ephemeral/667.sagetest

# two minutes timeout
try_cnt=40
while [[ ("$try_cnt" -gt "0") && ((! -f "$test_file") || ("$(cat $test_file)" != "$content")) ]]; do
  timeout -s SIGINT 1 bash -c "sage_user=$sage_user $curdir/auntie ^$curdir/${USER}_sage_testfile $glad_portal 667 sagetest" || echo -n ''
  echo "Trying to start Sage..."
  sleep 2
  try_cnt=$((try_cnt-1))
done
rm -f $test_file
}
[/quote]

Too long wait. If it does not work in 30 sec, report an error and stop.

[quote]
start_sage () {
# check whether sage is really stopped.
local tmp="$(status_sage)"
if [[ "$tmp" != "Sage is stopped." ]]; then
  echo "Sage is already started."
  echo ""
  echo "Status: "
  echo "$tmp"
  return 0
fi

remote_cmd "screen -dmS sage_service bash -c 'cd ~/sage; ./start.sh'"
wait_sage_start
echo "Sage is started."
}

Quote:stop sage

Code:

stop_sage () {
# stop sage
remote_cmd "cd ~/sage; ./stop.sh" > /dev/null

# kill all sage_service screens
remote_cmd "
for session in \$(screen -ls | grep 'sage_service' | grep -o '[0-9]\+\.' | grep -o '[0-9]\+')
do
    screen -S \${session} -p 0 -X quit
done
"
[/quote]

Looks good.

[quote]
# check whether sage is really stopped.
local tmp="$(status_sage)"
if [[ "$tmp" == "Sage is stopped." ]]; then
  echo "Sage is stopped."
else
  echo "Sage failed to stop."
  echo ""
  echo "Status: "
  echo "$tmp"
fi
}

Quote:status sage

Code:

status_sage () {
local output="$(remote_cmd "cd ~/sage; ./status.sh")"
if [[ "$output" == "Sage is stopped." ]]; then
  echo "Sage is stopped."
else
  echo "$output"
fi
}

(09-07-2016 05:35 PM)lingu Wrote:  This is too heavyweight -- it may even disrupt Sage if it is busy.

Sending a file takes 2-3s. This is considered long/heavyweight.

Quote:

Quote:Then glad operative user & sage_user can use sage_key to execute remote command to sage_user@sage_portal.

2. The glad_portal IP can be get from $gb/conf/config.sh

How (a convar like glad_portal?) Have we made sure such a varaible must exist in $gb/conf/config.sh. If not, please add a note there "need implement" so that we know to create a TODO item and implement this later after this design is endorsed.

Quote:What if the wait_sage_start fails?

We need a way to check Sage has started. I thought this is what I thought you were doing. But it looks like here if auntie continues to fail the wait still completes without an error halt.

(09-07-2016 05:53 PM)YU_Xinjie Wrote:  

(09-07-2016 05:35 PM)lingu Wrote:  This is too heavyweight -- it may even disrupt Sage if it is busy.

Sending a file takes 2-3s. This is considered long/heavyweight.

I ever tested to send a small file from s188 to s188. It only costs 0m0.009s. I think it is good enough.

Quote:We already have a $glad_portal that can be used directly. It is set by util/install.sh during installing.

Quote:

Quote:What if the wait_sage_start fails?

We need a way to check Sage has started. I thought this is what I thought you were doing. But it looks like here if auntie continues to fail the wait still completes without an error halt.

This is a bug. I will fix it.

(09-07-2016 06:18 PM)lingu Wrote:  Please revise the design of this part first as a reply then I can reply to endorse the overall design.

The real "fix" should be after the design is endorsed.

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  goal

1. the script can be used in both sage_portal & glad_portal to start/stop/status sage.
2. user can start sage before glad, and stop sage after glad.

The script would replace the current start.sh/stop.sh/status.sh in the future.
The script may inspire us about how to set up regression tests for sage.

For shbio's project, I have already written a script /thinker/dstore/run/sage to support a part of the goal. I would try to generalize it.

config design

The brief idea is to use auntie to send a file to glad_portal. Then check the file content.

1.
require sage_user can password-less login sage_user@sage_portal
copy private key of sage_user@sage_portal into $gb/conf/sage_key

Then glad operative user & sage_user can use sage_key to execute remote command to sage_user@sage_portal.

2. The glad_portal IP can be get from $gb/conf/config.sh

3.
create a config file $gb/conf/sage_portal_ip to record the IP of sage_portal. Or we can also write it into $gb/conf/config.sh.

4. let sage_user@sage_portal can password-less login glad_user@glad_portal.

start sage

Code:

wait_sage_start () {
content="This is test content."
echo $content > $curdir/${USER}_sage_testfile
test_file=/thinker/bin/ephemeral/667.sagetest

# 30 seconds timeout
try_cnt=10
while [[ ("$try_cnt" -gt "0") && ((! -f "$test_file") || ("$(cat $test_file)" != "$content")) ]]; do
  timeout -s SIGINT 1 bash -c "sage_user=$sage_user $curdir/auntie ^$curdir/${USER}_sage_testfile $glad_portal 667 sagetest" || echo -n ''
  echo "Trying to start Sage..."
  sleep 2
  try_cnt=$((try_cnt-1))
done
rm -f $test_file

if [[ "$try_cnt" == "0" ]];then
  # timeout
  return 1
else
  return 0
fi
}

start_sage () {
# check whether sage is really stopped.
local tmp="$(status_sage)"
if [[ "$tmp" != "Sage is stopped." ]]; then
  echo "Sage is already started."
  echo ""
  echo "Status: "
  echo "$tmp"
  return 0
fi

remote_cmd "screen -dmS sage_service bash -c 'cd ~/sage; ./start.sh'"

local rt="succ"
wait_sage_start || rt="fail"
if [[ "$rt" == "succ" ]]; then
  echo "Sage is started."
else
  echo "Sage fails to start."
fi
}

stop sage

Code:

stop_sage () {
# stop sage
remote_cmd "cd ~/sage; ./stop.sh" > /dev/null

# kill all sage_service screens
remote_cmd "
for session in \$(screen -ls | grep 'sage_service' | grep -o '[0-9]\+\.' | grep -o '[0-9]\+')
do
    screen -S \${session} -p 0 -X quit
done
"

# check whether sage is really stopped.
local tmp="$(status_sage)"
if [[ "$tmp" == "Sage is stopped." ]]; then
  echo "Sage is stopped."
else
  echo "Sage failed to stop."
  echo ""
  echo "Status: "
  echo "$tmp"
fi
}

status sage

Code:

status_sage () {
local output="$(remote_cmd "cd ~/sage; ./status.sh")"
if [[ "$output" == "Sage is stopped." ]]; then
  echo "Sage is stopped."
else
  echo "$output"
fi
}

(09-07-2016 06:37 PM)YU_Xinjie Wrote:  

(09-07-2016 06:18 PM)lingu Wrote:  Please revise the design of this part first as a reply then I can reply to endorse the overall design.

The real "fix" should be after the design is endorsed.

Updated the code of "start sage" in the headpost.

Please take a look again.

(09-07-2016 05:35 PM)lingu Wrote:  

Quote:wait_sage_start () {
content="This is test content."
echo $content > $curdir/${USER}_sage_testfile
test_file=/thinker/bin/ephemeral/667.sagetest

# two minutes timeout
try_cnt=40
while [[ ("$try_cnt" -gt "0") && ((! -f "$test_file") || ("$(cat $test_file)" != "$content")) ]]; do
timeout -s SIGINT 1 bash -c "sage_user=$sage_user $curdir/auntie ^$curdir/${USER}_sage_testfile $glad_portal 667 sagetest" || echo -n ''
echo "Trying to start Sage..."
sleep 2
try_cnt=$((try_cnt-1))
done
rm -f $test_file
}

Too long wait. If it does not work in 30 sec, report an error and stop.

(09-08-2016 04:48 PM)YU_Xinjie Wrote:  For the private key, I encounter an issue:

SSH command would check the premission of private key. If the key can readable by others, SSH would just ignore the key. One method to skip the check is to set the owner of key to be a user that would never use the key. For example:

Code:

[0][15:40:42] xinjie@devmac0e0:/thinker/dstore/gene/glad/conf
$ ll sage_key
-rw-r--r--. 1 dstore dstore 1679 Sep  8 14:57 sage_key

I set the owner to be dstore. Then user gene, sage, xinjie can use the key to access sage_portal. Only user dstore can not do that.

(09-07-2016 05:35 PM)lingu Wrote:  

Quote:1.
require sage_user can password-less login sage_user@sage_portal

This is fine if all are sage_user.

Quote:copy private key of sage_user@sage_portal into $gb/conf/sage_key

$gb is owned by glad_user and all operative users can access it. So this is a security vulnerability. OK now. But make a TODO to fix it later.

(09-15-2016 12:34 AM)lingu Wrote:  

(09-08-2016 04:48 PM)YU_Xinjie Wrote:  For the private key, I encounter an issue:

SSH command would check the premission of private key. If the key can readable by others, SSH would just ignore the key. One method to skip the check is to set the owner of key to be a user that would never use the key. For example:

Code:

[0][15:40:42] xinjie@devmac0e0:/thinker/dstore/gene/glad/conf
$ ll sage_key
-rw-r--r--. 1 dstore dstore 1679 Sep  8 14:57 sage_key

I set the owner to be dstore. Then user gene, sage, xinjie can use the key to access sage_portal. Only user dstore can not do that.

If the user is not the owner but can read the file and other users can read the key file, too, why does SSH allow the user to use the key? This seems to be a bug in SSH.

Is the behavior dependable? If the current behavior is not well defined, we may not want to rely on it although it works now. In the future, if the behavior changes, our programs will break.

Can we use multiple key files of the same content, i.e., key-sage, key-xinjie, ..., so that the key file's access can be restricted to that particular user?

In general, avoid hacks unless there is no other way out. Keep things simple stupid. Hacks are fun, but it's not dependable. It's like sleeping a drunk girl -- if anything lasted after the fun through the next sunrise it would be some mess of drying sperm mixed with vomits. That's not beauty of life.

(09-15-2016 12:49 AM)lingu Wrote:  

(09-07-2016 05:35 PM)lingu Wrote:  

Quote:1.
require sage_user can password-less login sage_user@sage_portal

This is fine if all are sage_user.

Quote:copy private key of sage_user@sage_portal into $gb/conf/sage_key

$gb is owned by glad_user and all operative users can access it. So this is a security vulnerability. OK now. But make a TODO to fix it later.

I think I may not have really understood what we are doing here. Are we trying to let glad_user ssh to sage_user@sage_portal? Or are we trying to let sage_user ssh back to sage_user@sage_portal?

For the latter, we can perhaps let all sage_user have the same key on all nodes? But the keys may not be stored in $gb.

For the former, we should perhaps add glad_user's pub key to sage_user@portal's authorized_keys during the installation of glad?

(09-18-2016 03:59 PM)YU_Xinjie Wrote:  Neither.

The glad operative users & sage_user need to ssh to sage_user@portal, in "sage" script.

(09-23-2016 10:41 PM)lingu Wrote:  What function needs this? Where is the designer's info? We should review it.

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  Then glad operative user & sage_user can use sage_key to execute remote command to sage_user@sage_portal.

Quote:1. the script can be used in both sage_portal & glad_portal to start/stop/status sage.
2. user can start sage before glad, and stop sage after glad.

(09-27-2016 11:33 AM)YU_Xinjie Wrote:  

(09-23-2016 10:41 PM)lingu Wrote:  What function needs this? Where is the designer's info? We should review it.

It is the original design in the headpost, which you already endorse it.

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  Then glad operative user & sage_user can use sage_key to execute remote command to sage_user@sage_portal.

It is needed by the goal:

Quote:1. the script can be used in both sage_portal & glad_portal to start/stop/status sage.
2. user can start sage before glad, and stop sage after glad.

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  goal

1. the script can be used in both sage_portal & glad_portal to start/stop/status sage.
2. user can start sage before glad, and stop sage after glad.

The script would replace the current start.sh/stop.sh/status.sh in the future.
The script may inspire us about how to set up regression tests for sage.

For shbio's project, I have already written a script /thinker/dstore/run/sage to support a part of the goal. I would try to generalize it.

config design

The brief idea is to use auntie to send a file to glad_portal. Then check the file content.

1.
require sage_user can password-less login sage_user@sage_portal
copy private key of sage_user@sage_portal into $gb/conf/sage_key

But SSH command would check the premission of private key. If the key can readable by others, SSH would just ignore the key. One method to skip the check is to set the owner of key to be a user that would never use the key. For example:

Code:

[0][15:40:42] xinjie@devmac0e0:/thinker/dstore/gene/glad/conf
$ ll sage_key
-rw-r--r--. 1 dstore dstore 1679 Sep  8 14:57 sage_key

I set the owner to be dstore. Then user gene, sage, xinjie can use the key to access sage_portal. Only user dstore can not do that.

Then glad operative user & sage_user can use sage_key to execute remote command to sage_user@sage_portal.

2. The glad_portal IP can be get from $gb/conf/config.sh

3.
create a config file $gb/conf/sage_portal_ip to record the IP of sage_portal. Or we can also write it into $gb/conf/config.sh.

4. let sage_user@sage_portal can password-less login glad_user@glad_portal.

start sage

Code:

wait_sage_start () {
content="This is test content."
echo $content > $curdir/${USER}_sage_testfile
test_file=/thinker/bin/ephemeral/667.sagetest

# 30 seconds timeout
try_cnt=10
while [[ ("$try_cnt" -gt "0") && ((! -f "$test_file") || ("$(cat $test_file)" != "$content")) ]]; do
  timeout -s SIGINT 1 bash -c "sage_user=$sage_user $curdir/auntie ^$curdir/${USER}_sage_testfile $glad_portal 667 sagetest" || echo -n ''
  echo "Trying to start Sage..."
  sleep 2
  try_cnt=$((try_cnt-1))
done
rm -f $test_file

if [[ "$try_cnt" == "0" ]];then
  # timeout
  return 1
else
  return 0
fi
}

start_sage () {
# check whether sage is really stopped.
local tmp="$(status_sage)"
if [[ "$tmp" != "Sage is stopped." ]]; then
  echo "Sage is already started."
  echo ""
  echo "Status: "
  echo "$tmp"
  return 0
fi

remote_cmd "screen -dmS sage_service bash -c 'cd ~/sage; ./start.sh'"

local rt="succ"
wait_sage_start || rt="fail"
if [[ "$rt" == "succ" ]]; then
  echo "Sage is started."
else
  echo "Sage fails to start."
fi
}

stop sage

Code:

stop_sage () {
# stop sage
remote_cmd "cd ~/sage; ./stop.sh" > /dev/null

# kill all sage_service screens
remote_cmd "
for session in \$(screen -ls | grep 'sage_service' | grep -o '[0-9]\+\.' | grep -o '[0-9]\+')
do
    screen -S \${session} -p 0 -X quit
done
"

# check whether sage is really stopped.
local tmp="$(status_sage)"
if [[ "$tmp" == "Sage is stopped." ]]; then
  echo "Sage is stopped."
else
  echo "Sage failed to stop."
  echo ""
  echo "Status: "
  echo "$tmp"
fi
}

status sage

Code:

status_sage () {
local output="$(remote_cmd "cd ~/sage; ./status.sh")"
if [[ "$output" == "Sage is stopped." ]]; then
  echo "Sage is stopped."
else
  echo "$output"
fi
}

(10-19-2016 06:37 PM)YU_Xinjie Wrote:  Updated the design of "sage status". Added the design of "sage ps".

(12-06-2016 12:32 PM)YU_Xinjie Wrote:  I suggest we add a "sage config VAR_NAME" command, so that app of Sage (such as GLAD) can query the config value easily.

Quote:It is needed for many situations:
1. We need to use "atp_cnt" to set up glad_size of GLAD.

Quote:2. GLAD needs the config.node info.

(12-06-2016 01:35 PM)lingu Wrote:  

(12-06-2016 12:32 PM)YU_Xinjie Wrote:  I suggest we add a "sage config VAR_NAME" command, so that app of Sage (such as GLAD) can query the config value easily.

We already have a way to let Sage store and report k/vs. Why can't we use the existing way?

Quote:

Quote:It is needed for many situations:
1. We need to use "atp_cnt" to set up glad_size of GLAD.

Where do we need to use glad_size? I am curious to know.

Quote:

Quote:2. GLAD needs the config.node info.

In which case?

(12-06-2016 03:32 PM)YU_Xinjie Wrote:  

(12-06-2016 01:35 PM)lingu Wrote:  

(12-06-2016 12:32 PM)YU_Xinjie Wrote:  I suggest we add a "sage config VAR_NAME" command, so that app of Sage (such as GLAD) can query the config value easily.

We already have a way to let Sage store and report k/vs. Why can't we use the existing way?

Do you mean "query Sage"? I am okay to use it. But I still do not know how to use it. Let us discuss it here: http://tab.d-thinker.org/showthread.php?tid=6433&pid=30046#pid30046

Quote:

Quote:

Quote:2. GLAD needs the config.node info.

In which case?

It is needed in the shuffle tmp implementation. It is okay to delay the config.node info fetching.

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  Users' info

http://tab.d-thinker.org/showthread.php?tid=5694

(01-12-2017 06:55 PM)lingu Wrote:  

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  Users' info

http://tab.d-thinker.org/showthread.php?tid=5694

That links to OMUD info... seems strange.

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  sage start

Start a sage_screen to run script start_sage.
start_sage script looks like this:

Quote:...

do_start() {
./detect_listen.sh &
make run, which would calls `dt run sage-svc $sage_base/config.pcf`
if return non-0 {
.. mayday_msg = "SAGE FATAL EXIT. "
.. check the stdout log of Sage. If it is because one of the task failure {
.... find the container (DT) log of this task
.... mayday_msg += fetch the task command line from the container log
.. }

.. #Do some clean work
.. kill (clean) the screen session of "sage wake"
.. kill (clean) the screen session of "sage start", namely the current session.
}
}
...

(06-02-2017 06:24 PM)YU_Xinjie Wrote:  @lingu Please review.

The driving example: http://tab.d-thinker.org/showthread.php?tid=8362&pid=41989#pid41989
We needs to let sage complains to mayday when sage fails.

The design is as follows:
[quote='YU_Xinjie' pid='11533' dateline='1468313653']
sage start

Start a sage_screen to run script start_sage.
start_sage script looks like this:

Quote:...

do_start() {
./detect_listen.sh &
make run, which would calls `dt run sage-svc $sage_base/config.pcf`
if return non-0 {
.. mayday_msg = "SAGE FATAL EXIT. "

It is okay to write a mayday log message on this case.

[quote]
.. check the stdout log of Sage. If it is because one of the task failure {
.... find the container (DT) log of this task
.... mayday_msg += fetch the task command line from the container log
.. }

Quote:.. #Do some clean work
.. kill (clean) the screen session of "sage wake"
.. kill (clean) the screen session of "sage start", namely the current session.
}[/u]
}
...

(05-08-2019 10:01 PM)lingu Wrote:  The sage program begins with the following logic.

Code:

base=current dir
LearnPcf $base/config.pcf
ips=$base/hosts.ips

I suggest we also learn /thinker/etc/soft/sage/mic.pcf but $base/config.pcf take precedence over /thinker/etc/soft/sage/mic.pcf

RR xwcwt

(05-09-2019 10:37 AM)xwcwt Wrote:  Also I feel it will be

Code:

modname = sage
LearnPcf /thinker/etc/soft/$modname/mic.pcf before LearnPcf $base/config.pcf

Update into head-post.

(05-09-2019 10:37 AM)xwcwt Wrote:  

(05-08-2019 10:01 PM)lingu Wrote:  The sage program begins with the following logic.

Code:

base=current dir
LearnPcf $base/config.pcf
ips=$base/hosts.ips

I suggest we also learn /thinker/etc/soft/sage/mic.pcf but $base/config.pcf take precedence over /thinker/etc/soft/sage/mic.pcf

RR xwcwt

Good to me.

Also I feel it will be

Code:

modname = sage
LearnPcf /thinker/etc/soft/$modname/mic.pcf before LearnPcf $base/config.pcf

Update into head-post.

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  sage main logiccommon var import

Code:

modname = sage
base=current dir
LearnPcf /thinker/etc/soft/$modname/mic.pcf
LearnPcf $base/config.pcf
ips=$base/hosts.ips

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  sage main logiccommon var import

modname = sage
base=current dir
LearnPcf /thinker/etc/soft/$modname/mic.pcf
LearnPcf $base/config.pcf
ips=$base/hosts.ips
set timeout to be half of the number of IPs

(05-09-2019 07:09 PM)lingu Wrote:  

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  sage main logiccommon var import

modname = sage
base=current dir
LearnPcf /thinker/etc/soft/$modname/mic.pcf
LearnPcf $base/config.pcf
ips=$base/hosts.ips
set timeout to be half of the number of IPs

Driver: The IP counting is based on 'wc -l $ips'. But $ips may not exist on non-portal.

I suggest we use a plain number 3 if $ips does not exist.

RR xwcwt

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  sage tasks
cat $sage_base/stdout/tasks_graph.tmp.<timestamp>.

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  sage tasks
sage tasks invokes function tasks_sage():
local tasks_graph_file=$base/stdout/tasks_graph.tmp
while True {
while True {

Quote: cnt = count ${tasks_graph_file}*;
sleep 5 if cnt>1 || cnt==0;
otherwise break;
}
check timestamp;
cat $sage_base/stdout/tasks_graph.tmp.<timestamp>
update timestamp;
}
[/code]

(05-10-2019 07:46 PM)lingu Wrote:  driver: we used to rely on a sage awake screen to collect sage tasks result, but that screen runs only on sage_portal. Now we allow non-sage_portal to interact with sage. So we should run 'auntie' to wake up sage before collecting the sage tasks result.

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  sage tasks
sage tasks invokes function tasks_sage():
local tasks_graph_file=$base/stdout/tasks_graph.tmp
while True {
while True {

Proposal: add 'auntie' here

Quote: cnt = count ${tasks_graph_file}*;
sleep 5 if cnt>1 || cnt==0;
otherwise break;
}
check timestamp;
cat $sage_base/stdout/tasks_graph.tmp.<timestamp>
update timestamp;
}
[/code]

RR xwcwt

(07-12-2016 05:54 PM)YU_Xinjie Wrote:  sage tasks
sage tasks invokes function tasks_sage():

Code:

local tasks_graph_file=$base/stdout/tasks_graph.tmp
while True {
  while True {
    cnt = count ${tasks_graph_file}*;
    sleep 5 if cnt>1 || cnt==0;
    otherwise break;
  }
  call auntie ##To wake up sage to collect info in all nodes.
  check timestamp;
  cat $sage_base/stdout/tasks_graph.tmp.<timestamp>
  update timestamp;
}

(05-10-2019 07:56 PM)lingu Wrote:  Driver: Sage configuration can be quite complex. It is often convenient to evaluate a var in Sage either on command line or in a program. GLAD has a command 'gladinfo var' or 'glad info var' to read the value of a variable var. But Sage does not have it.

Proposal: add a command 'info' like the 'info' command in glad, an let 'sage info var' report the value of var.

RR zheny

(11-23-2019 11:48 PM)lingu Wrote:  

(05-10-2019 07:56 PM)lingu Wrote:  Driver: Sage configuration can be quite complex. It is often convenient to evaluate a var in Sage either on command line or in a program. GLAD has a command 'gladinfo var' or 'glad info var' to read the value of a variable var. But Sage does not have it.

Proposal: add a command 'info' like the 'info' command in glad, an let 'sage info var' report the value of var.

RR zheny

?

(11-24-2019 06:09 PM)zheny Wrote:  

(11-23-2019 11:48 PM)lingu Wrote:  

(05-10-2019 07:56 PM)lingu Wrote:  Driver: Sage configuration can be quite complex. It is often convenient to evaluate a var in Sage either on command line or in a program. GLAD has a command 'gladinfo var' or 'glad info var' to read the value of a variable var. But Sage does not have it.

Proposal: add a command 'info' like the 'info' command in glad, an let 'sage info var' report the value of var.

RR zheny

?

good to me
I suggest to update these info to head-post
---
sage info
sage info [var]: report the value of a variable $var
supported $var including
ips: nodes managed by sage
apt_cnt: current atp count
---

RR xwcwt
I will implement these a little later