cup.services package¶
Submodules¶
cup.services.autowait module¶
description: | auto wait related modules. |
---|
-
cup.services.autowait.
wait_until_file_exist
(dst_path, file_name, max_wait_sec=10, interval_sec=2, recursive=False)[source]¶ wait util the file exists or the function timeout
Parameters: - dst_path – searching path
- file_name – filename, support *
- max_wait_sec – max wating time until timeout
- interval_sec – check interval
- recursive – recursively search or not
Returns: True if found.
-
cup.services.autowait.
wait_until_reg_str_exist
(dst_file_path, reg_str, max_wait_sec=10, interval_sec=0.5)[source]¶ wait until any line in the file matches the reg_str(regular expression string)
Parameters: - dst_file_path – searching path
- reg_str – regular expression string
- max_wait_sec – maximum waiting time until timeout
- interval_sec – state check interval
Returns: True if found
-
cup.services.autowait.
wait_until_process_not_exist
(process_path, max_wait_sec=10, interval_sec=0.5)[source]¶ wait until the process does not exist anymore or the function timeouts
Parameters: - process_path – process cwd
- max_wait_sec – maximum waiting time until timeout. 10 seconds by default
- interval_sec – state check interval, 0.5 second by default
Returns: return True if the process disapper before timeout
-
cup.services.autowait.
wait_until_port_used
(port, max_wait_sec=10, interval_sec=0.5)[source]¶ wait until the port is used. Notice this function will invoke a bash shell to execute command [netstat]!
Returns: return True if the port is used
-
cup.services.autowait.
wait_until_process_used_ports
(process_path, ports, max_wait_sec=10, interval_sec=0.5)[source]¶ wait until the process has taken the ports before timeouts
Returns: True if all ports are used by the specific process. False, otherwise
-
cup.services.autowait.
wait_until_port_not_used
(port, max_wait_sec=10, interval_sec=0.5)[source]¶ wait until the port is free
Returns: return True if the port is free before timeout
-
cup.services.autowait.
wait_until_process_exist
(process_path, max_wait_sec=10, interval_sec=0.5)[source]¶ wait until the process exists
Parameters: - process_path – the specific process working path
- max_wait_sec – maximum waiting time until timeout
- interval_sec – state check interval
Returns: return True if the process is found before timeout
-
cup.services.autowait.
wait_until_process_killed
(process_path, ports, max_wait_sec=10, interval_sec=0.5)[source]¶ wait until the [process] does not exists and all [ports] are free
Parameters: - process_path – process cwd
- ports – port list
- interval_sec – state check interval
Returns: True if all conditions meet.
cup.services.buffers module¶
description: | buffer pool |
---|
-
class
cup.services.buffers.
Buffer
(items, block_size, uniqid)[source]¶ Bases:
object
Buffer object which you get from BufferPool.allocate(num)
cup.services.executor module¶
description: |
|
---|
-
class
cup.services.executor.
ExecutionService
(delay_exe_thdnum=3, queue_exec_thdnum=4)[source]¶ Bases:
object
execution service
-
delay_exec
(delay_time_insec, function, urgency, *args, **kwargs)[source]¶ delay_execute function after delay_time seconds
You can use urgency := executor.URGENCY_NORMAL, by default
TODO: consider about stopping timers when invoking stop function
-
cup.services.generator module¶
class CGeneratorMan(object)¶
Generate unique integers, strings and auto incremental uint. Notice CGeneratorMan is a singleton class, which means cup will keep only 1 instance per process.
init: |
|
---|---|
methods: |
|
cup.services.heartbeat module¶
description: | heartbeat related module |
---|
-
class
cup.services.heartbeat.
Device
(name)[source]¶ Bases:
object
Base class for all devices in heartbeat service
-
class
cup.services.heartbeat.
HeartbeatService
(judge_lost_in_sec, keep_lost=False)[source]¶ Bases:
object
HeartBeat service
-
class
cup.services.heartbeat.
LinuxHost
(name, init_this_host=False, iface='eth0', port=0)[source]¶ Bases:
cup.services.heartbeat.Device
a linux host resource
-
set_ip_port
(ipaddr)[source]¶ set ip information
Parameters: ipaddr – ipaddr should be string and something like 10.10.10.1
-
set_linux_res_bydict
(info_dict)[source]¶ - {
- ‘iface’: ‘eth0’, ‘ipaddr’: ‘10.10.10.1’, ‘port’: 8089, ‘cpu_idle’: 50, ‘mem_inuse’: 1024, # MB ‘mem_total’: 8192, ‘net_in’: 8192, # kb ‘net_out’: 102400, # kb
}
-
cup.services.msgbroker module¶
description: | Msg Broker Service. Every component of a process can produce_msg |
---|
-
class
cup.services.msgbroker.
BrokerCenter
(name)[source]¶ Bases:
cup.services.msgbroker.BaseBroker
Errmsg broker center
-
class
cup.services.msgbroker.
SystemErrmsgBroker
(name)[source]¶ Bases:
cup.services.msgbroker.BrokerCenter
system errmsg broker, you can use it to determine whether exiting from the system is on the way
-
register_msgtype_callback
(path, msg_type, callback_func)[source]¶ register msgtype with callback functions
-
cup.services.serializer module¶
description: | serilizers including local file serilizer |
---|
-
class
cup.services.serializer.
LogRecord
(length, log_id, log_type, log_mode, log_binary)¶ Bases:
tuple
-
length
¶ Alias for field number 0
-
log_binary
¶ Alias for field number 4
-
log_id
¶ Alias for field number 1
-
log_mode
¶ Alias for field number 3
-
log_type
¶ Alias for field number 2
-
-
class
cup.services.serializer.
LocalFileSerilizer
(storage_dir, skip_badlog=False, max_logfile_size=65536)[source]¶ Bases:
cup.services.serializer.BaseSerilizer
local file serilizer
-
open4write
(truncate_last_failure=True)[source]¶ Raises: Exception – if encounter any IOError, will raise IOError(errmsg)
-
purge_data
(before_logid)[source]¶ log files which contains log (less than before_logid) will be purged.
-
read
(record_num=128)[source]¶ load log into memory
Notice: If skip_badlog is not True, will raise IOError if the stream encounters any error.
Otherwise, the stream will skip the bad log file, move to next one and continue reading
Returns: - return a list of “record_num” of LogRecords.
b. If the count number of list is less than record_num, it means the stream encounter EOF, plz read again afterwards.
- If the returned is None, it means the stream got nothing, plz
- try again.
-
cup.services.threadpool module¶
description: | Guannan back-ported threadpool from twisted.python. if any concern, plz contact Guannan (mythmgn@gmail.com) |
---|---|
license: |
|
-
class
cup.services.threadpool.
ThreadPool
(minthreads=5, maxthreads=20, name=None, daemon_threads=False)[source]¶ Bases:
object
Threadpool class
-
add_1job
(func, *args, **kwargs)[source]¶ Add one job that you want the pool to schedule. Notice if you need to handle data after finishing [func], plz use [add_1job_with_callback] which supports a [callback] option.
Parameters: - func – function that will be scheduled by the thread pool
- *args –
args that the [func] needs
- **kw –
kwargs that [func] needs
-
add_1job_with_callback
(result_callback, func, *args, **kwargs)[source]¶ Parameters: - result_callback –
plz notice whether succeed or fail, the result_callback function will be called after [func] is called.
function result_callback needs to accept two parameters: (ret_in_bool, result). (True, result) will be passed to the [func] on success. (False, result) will be passed otherwise.
- if [func] raise any Exception, result_callback will get (False,
- failure_info) as well.
- func – same to func for add_1job
- *args –
args for [func]
- **kwargs –
kwargs for [func]
- result_callback –
-
dump_stats
(print_stdout=False)[source]¶ Dump the threadpool stat to log or stdout. Info is from class method [get_stats]
-
get_stats
()[source]¶ get threadpool running stats waiters_num is pending thread num working_num is working thread num thread_num is the total size of threads
- ::
- stat = {} stat[‘queue_len’] = self._jobqueue.qsize() stat[‘waiters_num’] = len(self._waiters) stat[‘working_num’] = len(self._working) stat[‘thread_num’] = len(self._threads)
-
Module contents¶
@author Guannan Ma @mythmgn