Home > Distfiles, Dowloader, Fetcher, Gentoo, GSOC, Linux, Packages, Portage > WR#4 – “Growing Muscles on Skeleton”

WR#4 – “Growing Muscles on Skeleton”

Fresh milk and cottage cheese gradually enforce the skeleton, at the same time growing muscles on it can take a lot of efforts. Nevertheless pulling-ups, mirror benchmarking [1], and other things were added to my program.

NOTE:
Previous weeks i kept writing reports, but while sending them i used my default email account, which unfortunately happened to be not the one subscribed to [gentoo-soc]. Hence, i don't think anybody got them.

Yesterday i resent these reports (better later then never):
- Project IDFetch - Weekly report #2 ("Replacing Stuff")
- Project IDFetch - Weekly report #3 ("Strawberry Issues")

Since i've already tried writing reports without sending them, why not to do the opposite - to send a report without actually writing it ;o)
So please take a look at the list of the options implemented in segget during the 4th week of IDFetch project.

IDFetch: adding muscles to segget.conf skeleton:
================================================

[folders]
# DISTFILES_DIR
# Define a dir to store distfiles
# Default:
# distfiles_dir=./distfiles
distfiles_dir=./distfiles

# SEGMENTS_DIR
# Define a dir to store distfiles' segments
# Default:
# segments_dir=./tmp
segments_dir=./tmp

[distfiles]
# MAX_CONNECTION_NUM_PER_DISTFILE
# Each distfile can have up to max_connection_num_per_distfile
# simultaneous connections.
# default:
# max_connection_num_per_distfile=3
max_connection_num_per_distfile=3

[segments]
# MAX_SEGMENT_SIZE
# Define maximum segment size in bytes.
# Default:
# max_segment_size=500000
max_segment_size=500000

# SYNOPSIS: resume_on=0 | 1
# - If resume_on set to 1:
# Before starting downloading a segment segget checks whether this
# segment is already downloaded, checks segments size and if size
# mathces considers this segment to be downloaded and skips downloading
# process.
# - If resume_on set to 0:
# Segget always starts new fetch for a segment regardless of the
# fact whether it is downloaded or not.
# Default:
# resume_on=1
resume_on=1

# MAX_TRIES
# If segment download was unsuccessful, new attempts are made. When
# attempts number reaches max_tries, segment gets FAILED status and
# error logged to error_log.
# Default:
# max_tries=10
max_tries=10

[connections]
# MAX_CONNECTIONS
# Define maximum number of connections
max_connections=10

# CONNECTION_TIMEOUT
# Set the number of seconds to wait while trying to connect. Use 0 to
# wait indefinitely. Pass a long. It should contain the maximum time in
# seconds that you allow the connection to the server to take. This
# only limits the connection phase, once it has connected, this option
# is of no more use. Set to zero to disable connection timeout (it will
# then only timeout on the system's internal timeouts). See also the
# TIMEOUT option.
# Default:
# connection_timeout=15
connection_timeout=15

# FTP_RESPONSE_TIMEOUT
# Set a timeout period (in seconds) on the amount of time that the
# server is allowed to take in order to generate a response message for
# a command before the session is considered hung. While awaiting for a
# response, this value overrides TIMEOUT. It is recommended that if
# used in conjunction with TIMEOUT, you set FTP_RESPONSE_TIMEOUT to a
# value smaller than TIMEOUT.
# Default:
# ftp_response_timeout=180
ftp_response_timeout=180

# TIMEOUT
# maximum amount of time to download segment in seconds
# Set the maximum number of seconds for a connection to execute.
# Pass a long as parameter containing the maximum time in seconds that
# you allow the transfer operation to take. Normally, name lookups can
# take a considerable
# time and limiting operations to less than a few minutes risk aborting
# perfectly normal operations.
# Default:
# timeout=500
timeout=500

# LOW_CONNECTION_SPEED_LIMIT
# Define the low speed limit for connection. Pass a long as parameter.
# It contains the transfer speed in bytes per second that the transfer
# should be below during LOW_CONNECTION_SPEED_TIME seconds to consider
# it too slow and abort.
# Default:
# low_connection_speed_limit=1000
low_connection_speed_limit=1000

# LOW_CONNECTION_SPEED_TIME
# Pass a long as parameter. It contains the time in seconds that the
# transfer should be below the LOW_CONNECTION_SPEED_LIMIT to consider
# it too slow and abort.
# Default:
# low_connection_speed_time=10
low_connection_speed_time=10

# MAX_CONNECTION_SPEED
# If a download exceeds this speed (counted in bytes per second) on
# cumulative average during the transfer, the transfer will pause to
# keep the average rate less than or equal to the parameter value.
# Defaults to unlimited speed.
# Default:
# max_connection_speed=0
max_connection_speed=0

# BIND INTERFACE / IP
# Pass a string as parameter. This sets the interface name to use as
# outgoing network interface. The name can be an interface name, an IP
# address, or a host name. No binding is set by default.
# Default:
# bind_interface=none
bind_interface=none

[mirrors]
# MAX_CONNECTIONS_NUM_PER_MIRROR
# Define how many simultaneous downloads from one mirror segget is
# allowed to have. While choosing a mirror segget will skip mirrors
# with max_connections_num_per_mirror active downloads.
# Default:
# max_connections_num_per_mirror=1
max_connections_num_per_mirror=1

# SYNOPSIS: collect_benchmark_stats_on=0 | 1
# - If set to 1, stats on mirrors performance will be collected.
# default:
# collect_benchmark_stats_on=1
# ***Note: at the moment collect_benchmark_stats_on can NOT be set to 0
collect_benchmark_stats_on=1

# SYNOPSIS: use_benchmark_stats=0 | 1
# If use_benchmark_stats=1 statistics on mirrors is used to rate them
# and therefore improve performance.
# Each time connection from a particular mirror closes mirror->dld_time,
# and mirror->dld_size get increased (in case of unsuccessful connection
# only time gets increased), so avg speed for a mirror can be
# calculated:
#
#           mirror->avg_speed=mirror->dld_size/mirror->dld_time.       (1)
#
# When new segment is going to be started segget goes through the list of
# the mirrors distfile/segment has, and asks each mirror for self_rating:
#
#          "ulong self_rating=mirror->mirror_on_the_wall();".
#
# This way segget chooses a mirror with the best self_rating. To calculate
# self_rating mirrors use the following formula:
#
#               self_rating=dld_time/dld_size*honesty.                 (2)
#
# So mirrors actually say how bad they are.
# Even mirrors can have critical times, so to give mirrors another chance
# honesty was added to the formula (2). honesty can get values in
# interval (0,1]. Each time connection from a mirror opens or closes mirror
# sets its honesty=1. If mirror was asked for self_evaluation with
# mirror->mirror_on_the_wall(), but wasn't chosen its honesty decreases
# somewhat (see [mirror].benchmark_oblivion option), so next time it will
# lie little bit more about how bad it's.
# Default:
# use_benchmark_results=1
# ***Note: at the moment use_benchmark_results can NOT be set to 0
use_benchmark_stats=1

# BENCHMARK_OBLIVION
# benchmark_oblivion option allows to adjust how fast segget "forgets"
# benchmarking statistics on mirrors performance.
# Each time mirror->mirror_on_the_wall() called, mirror decreases its
# honesty (to have more chances next time) using the following formula:
#
#       honesty=honesty*100/(100+settings.benchmark_oblivion)         (3)
#
# Therefore, setting benchmark_oblivion=100 will make mirror look twice
# less bad next time mirror->mirror_on_the_wall() called.
# Default:
# benchmark_oblivion=5
benchmark_oblivion=5

[user-data]
# USER_AGENT
# Set the User-Agent: header in the http request sent to the remote
# server.
# This can be used to fool servers or scripts.
# Default:
# user_agent=segget
user_agent=segget

[proxy]
# PROXY_IP_OR_NAME
# Specify a proxy to use (address and port).
# Set HTTP proxy to use. The parameter should be a string holding the
# proxy host name or dotted IP address. To specify port number in this
# string, append :[port] to the end of the host name. The proxy string
# may be prefixed with [protocol]:// since any such prefix will be
# ignored. The proxy's port number may optionally be specified with the
# separate option. If not specified, by default port 1080 will be used
# for proxies. When you tell segget to use an HTTP proxy, segget will
# transparently convert operations to HTTP even if you specify an FTP
# URL etc. Segget respects the environment variables http_proxy,
# ftp_proxy, all_proxy etc, if any of those are set. The PROXY option
# does however override any possibly set environment variables.
# Default:
# proxy_ip_or_name=none
proxy_ip_or_name=none

# PROXY_PORT
# Set the proxy port to connect to unless it is specified in the PROXY
# option.
# Default:
# proxy_port=3128
proxy_port=3128

# PROXY_USER
# Set user name to use for the transfer while connecting to Proxy.
# The PROXY_USER option should be used in same way as the
# PROXY_PASSWORD is used. In order to specify the password to be used
# in conjunction with the user name use the PROXY_PASSWORD option.
# Default:
# proxy_user=none
proxy_user=none

# PROXY_PASSWORD
# Set password to use for the transfer while connecting to Proxy.
# The PROXY_PASSWORD option should be used in conjunction with the
# PROXY_USER option.
# Default:
# proxy_password=none
proxy_password=none

# SYNOPSIS: proxy_off=0 | 1
# Setting the proxy_off=1 will explicitly disable the use of a proxy,
# even if there is an environment variable set for it.
# Default:
# proxy_off=1
proxy_off=1

[logs]
# LOGS_DIR
# Define a dir to store log files.
# Default:
# logs_dir=./logs
logs_dir=./logs

# GENERAL_LOG_FILE
# Define a file name to store general log.
# Default:
# general_log_file=segget.log
general_log_file=segget.log

# ERROR_LOG_FILE
# Define a file name to store error log.
# Default:
# error_log_file=segget.log
error_log_file=error.log

# DEBUG_LOG_FILE
# Define a file name to store debug log.
# Default:
# debug_log_file=segget.log
debug_log_file=debug.log

[1]http://www.twolia.com/blogs/daily-beauty-break/files/2010/04/mirror-on-the-wall.jpg

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: