Package ec.eval

# Class MasterProblem

java.lang.Object
ec.Problem
ec.eval.MasterProblem

All Implemented Interfaces:
  `ec.co.ConstructiveProblemForm`, `GroupedProblemForm`, `Prototype`, `Setup`, `SimpleProblemForm`, `Serializable`, `Cloneable`

public class MasterProblem extends Problem implements SimpleProblemForm, GroupedProblemForm, ec.co.ConstructiveProblemForm
MasterProblem.java

The MasterProblem is a special ECJ problem that performs evaluations by sending them to a remote Slave process to be evaluated. As it implements both the SimpleProblemForm and the GroupedProblemForm interfaces, the MasterProblem can perform both traditional EC evaluations as well as coevolutionary evaluations.

When a MasterProblem is specified by the Evaluator, the Problem is set up as usual, but then the MasterProblem replaces it. The Problem is not garbage collected -- instead, it's hung off the MasterProblem's `problem` variable. In some sense the Problem is "pushed aside".

If the Evaluator begins by calling prepareToEvaluate(), and we're not doing coevolution, then the MasterProblem does not evaluate individuals immediately. Instead, it waits for at most jobSize individuals be submitted via evaluate(), and then sends them all off in a group, called a job, to the remote slave. In other situations (coevolution, or no prepareToEvaluate()) the MasterProblem sends off individuals immediately.

It may be the case that no Slave has space in its queue to accept a new job containing, among others, your new individual. In this case, calling evaluate() will block until one comes available. You can avoid this by testing for availability first by calling canEvaluate(). Note that canEvaluate() and evaluate() together are not atomic and so you should not rely on this facility if your system uses multiple threads.

When the individuals or their fitnesses return, they are immediately updated in place. You have three options to wait for them:

- You can wait for all the individuals to finish evaluation by calling finishEvaluating(). If you call this method before a job is entirely filled, it will be sent in truncated format (which generally is perfectly fine). You then block until all the jobs have been completed and the individuals updated.

- You can block until at least one individual is available, by calling getNextEvaluatedIndividual(), which blocks and then returns the individual that was just completed.

- You can test in non-blocking fashion to see if an individual is available, by calling evaluatedIndividualAvailable(). If this returns true, you may then call getNextEvaluatedIndividual() to get the individual. Note that this isn't atomic, so don't use it if you have multiple threads.

Parameters

base.`debug-info` boolean | (whether the system should display information useful for debugging purposes)
base.`job-size` integer > 0 | (how large should a job be at most?)
`eval.master.port` int | (the port where the slaves will connect)
`eval.compression` boolean | (whether the communication with the slaves should be compressed or not)
`eval.masterproblem.max-jobs-per-slave` int | (the maximum load (number of jobs) per slave at any point in time)

See Also:
  - Serialized Form

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `boolean`
  `batchMode`
  `SlaveMonitor`
  `monitor`
  `static final String`
  `P_DEBUG_INFO`
  `static final String`
  `P_JOB_SIZE`
  `Problem`
  `problem`

  ### Fields inherited from class ec.Problem

  `P_PROBLEM`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `MasterProblem()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `boolean`
  `canEvaluate()`
  Asynchronous Steady-State EC only: Returns true if the problem is ready to evaluate.
  `Object`
  `clone()`
  Creates a new individual cloned from a prototype, and suitable to begin use in its own evolutionary context.
  `void`
  `closeContacts(EvolutionState state, int result)`
  Gracefully close contacts with the slaves
  `void`
  `describe(EvolutionState state, Individual ind, int subpopulation, int threadnum, int log)`
  Part of SimpleProblemForm.
  `void`
  `evaluate(EvolutionState state, Individual[] inds, boolean[] updateFitness, boolean countVictoriesOnly, int[] subpops, int threadnum)`
  Evaluates the individuals found in ind together.
  `void`
  `evaluate(EvolutionState state, Individual ind, int subpopulation, int threadnum)`
  Evaluates the individual in ind, if necessary (perhaps not evaluating them if their evaluated flags are true), and sets their fitness appropriately.
  `boolean`
  `evaluatedIndividualAvailable()`
  This will only return true if (1) the EvolutionState is a SteadyStateEvolutionState and (2) an individual has returned from the system.
  `void`
  `finishEvaluating(EvolutionState state, int threadnum)`
  Will be called by the Evaluator after prepareToEvaluate(...) is called and then a series of individuals are evaluated.
  `List<ec.co.Component>`
  `getAllComponents()`
  `List<ec.co.Component>`
  `getAllowedComponents(ec.co.ConstructiveIndividual partialSolution)`
  Return the allowable "neighborhood" of components that
  `ec.co.Component`
  `getArbitraryComponent(EvolutionState state, int thread)`
  Chooses an arbitrary component from the problem domain.
  `ec.co.Component`
  `getComponentFromString(String s)`
  Decode a String representation of a solution component.
  `QueueIndividual`
  `getNextEvaluatedIndividual()`
  This method blocks until an individual is available from the slaves (which will cause evaluatedIndividualAvailable() to return true), at which time it returns the individual.
  `void`
  `initializeContacts(EvolutionState state)`
  Initialize contacts with the slaves
  `boolean`
  `isCompleteSolution(ec.co.ConstructiveIndividual solution)`
  Determine whether a given specifies a complete solution to this problem.
  `boolean`
  `isGroupedProblem()`
  Returns true if the underlying class returns true for isGroupedProblem()
  `boolean`
  `isViolated(ec.co.ConstructiveIndividual partialSolution, ec.co.Component component)`
  `int`
  `numComponents()`
  `int`
  `postprocessPopulation(EvolutionState state, Population pop, boolean[] assessFitness, boolean countVictoriesOnly)`
  Finish processing the population (such as fitness information) after evaluation.
  `void`
  `prepareToEvaluate(EvolutionState state, int threadnum)`
  May be called by the Evaluator prior to a series of individuals to evaluate, and then ended with a finishEvaluating(...).
  `void`
  `preprocessPopulation(EvolutionState state, Population pop, boolean[] prepareForFitnessAssessment, boolean countVictoriesOnly)`
  Set up the population `pop` (such as fitness information) prior to evaluation.
  `void`
  `receiveAdditionalData(EvolutionState state, DataInputStream dataIn)`
  This method is called on a MasterProblem by the Slave.
  `void`
  `reinitializeContacts(EvolutionState state)`
  Reinitialize contacts with the slaves
  `void`
  `sendAdditionalData(EvolutionState state, DataOutputStream dataOut)`
  This method is called from the SlaveMonitor's accept() thread to optionally send additional data to the Slave via the dataOut stream.
  `void`
  `setup(EvolutionState state, Parameter base)`
  Sets up the object by reading it from the parameters stored in state, built off of the parameter base base.
  `void`
  `transferAdditionalData(EvolutionState state)`
  This method is called by a Slave to transfer data previously loaded via receiveAdditionalData() to a running EvolutionState at the beginning of evolution.

  ### Methods inherited from class ec.Problem

  `defaultBase, describe`

  ### Methods inherited from class java.lang.Object

  `equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### P_DEBUG_INFO

    public static final String P_DEBUG_INFO
    See Also:
      - Constant Field Values

  - ### P_JOB_SIZE

    public static final String P_JOB_SIZE
    See Also:
      - Constant Field Values

  - ### batchMode

    public boolean batchMode

  - ### monitor

    public transient SlaveMonitor monitor

  - ### problem

    public Problem problem

- ## Constructor Details

  - ### MasterProblem

    public MasterProblem()

- ## Method Details

  - ### clone

    public Object clone()
    Description copied from interface: `Prototype`
    Creates a new individual cloned from a prototype, and suitable to begin use in its own evolutionary context.

    Typically this should be a full "deep" clone. However, you may share certain elements with other objects rather than clone hem, depending on the situation:

    - If you hold objects which are shared with other instances, don't clone them.
    - If you hold objects which must be unique, clone them.
    - If you hold objects which were given to you as a gesture of kindness, and aren't owned by you, you probably shouldn't clone them.
    - DON'T attempt to clone: Singletons, Cliques, or Populations, or Subpopulation.
    - Arrays are not cloned automatically; you may need to clone an array if you're not sharing it with other instances. Arrays have the nice feature of being copyable by calling clone() on them.

    Implementations.

    - If no ancestor of yours implements clone(), and you have no need to do clone deeply, and you are abstract, then you should not declare clone().
    - If no ancestor of yours implements clone(), and you have no need to do clone deeply, and you are not abstract, then you should implement it as follows:

      ``

           public Object clone()
               {
               try
                   {
                   return super.clone();
                   }
               catch ((CloneNotSupportedException e)
                   { throw new InternalError(); } // never happens
               }

    - If no ancestor of yours implements clone(), but you need to deep-clone some things, then you should implement it as follows:

      ``

           public Object clone()
               {
               try
                   {
                   MyObject myobj = (MyObject) (super.clone());

                   // put your deep-cloning code here...
                   }
               catch ((CloneNotSupportedException e)
                   { throw new InternalError(); } // never happens
               return myobj;
               }

    - If an ancestor has implemented clone(), and you also need to deep clone some things, then you should implement it as follows:

      ``

           public Object clone()
               {
               MyObject myobj = (MyObject) (super.clone());

               // put your deep-cloning code here...

               return myobj;
               }

    Specified by:
      `clone` in interface `Prototype`
    Overrides:
      `clone` in class `Problem`

  - ### setup

    public void setup(EvolutionState state, Parameter base)
    Description copied from interface: `Prototype`
    Sets up the object by reading it from the parameters stored in state, built off of the parameter base base. If an ancestor implements this method, be sure to call super.setup(state,base); before you do anything else.

    For prototypes, setup(...) is typically called once for the prototype instance; cloned instances do not receive the setup(...) call. setup(...) may be called more than once; the only guarantee is that it will get called at least once on an instance or some "parent" object from which it was ultimately cloned.

    Specified by:
      `setup` in interface `Prototype`
    Specified by:
      `setup` in interface `Setup`
    Overrides:
      `setup` in class `Problem`

  - ### prepareToEvaluate

    public void prepareToEvaluate(EvolutionState state, int threadnum)
    Description copied from class: `Problem`
    May be called by the Evaluator prior to a series of individuals to evaluate, and then ended with a finishEvaluating(...). If this is the case then the Problem is free to delay modifying the individuals or their fitnesses until at finishEvaluating(...). If no prepareToEvaluate(...) is called prior to evaluation, the Problem must complete its modification of the individuals and their fitnesses as they are evaluated as stipulated in the relevant evaluate(...) documentation for SimpleProblemForm or GroupedProblemForm. The default method does nothing. Note that prepareToEvaluate() can be called *multiple times* prior to finishEvaluating() being called -- in this case, the subsequent calls may be ignored.
    Overrides:
      `prepareToEvaluate` in class `Problem`

  - ### finishEvaluating

    public void finishEvaluating(EvolutionState state, int threadnum)
    Description copied from class: `Problem`
    Will be called by the Evaluator after prepareToEvaluate(...) is called and then a series of individuals are evaluated. However individuals may be evaluated without prepareToEvaluate or finishEvaluating being called at all. See the documentation for prepareToEvaluate for more information. The default method does nothing.
    Overrides:
      `finishEvaluating` in class `Problem`

  - ### evaluate

    public void evaluate(EvolutionState state, Individual ind, int subpopulation, int threadnum)
    Description copied from interface: `SimpleProblemForm`
    Evaluates the individual in ind, if necessary (perhaps not evaluating them if their evaluated flags are true), and sets their fitness appropriately.
    Specified by:
      `evaluate` in interface `SimpleProblemForm`

  - ### describe

    public void describe(EvolutionState state, Individual ind, int subpopulation, int threadnum, int log)
    Description copied from class: `Problem`
    Part of SimpleProblemForm. Included here so you don't have to write the default version, which usually does nothing.
    Specified by:
      `describe` in interface `SimpleProblemForm`
    Overrides:
      `describe` in class `Problem`

  - ### preprocessPopulation

    public void preprocessPopulation(EvolutionState state, Population pop, boolean[] prepareForFitnessAssessment, boolean countVictoriesOnly)
    Description copied from interface: `GroupedProblemForm`
    Set up the population `pop` (such as fitness information) prior to evaluation. Although this method is not static, you should not use it to write to any instance variables in the GroupedProblem instance; this is because it's possible that the instance used is in fact the prototype, and you will have no guarantees that your instance variables will remain valid during the evaluate(...) process. Do not assume that `pop` will be the same as `state.pop` -- it may not. `state` is only provided to give you access to EvolutionState features. Typically you'd use this method to set the Fitness values of all Individuals to 0.

    countVictoriesOnly will be set if Individuals' fitness is to be based on whether they're the winner of a test, instead of based on the specifics of the scores in the tests. This really only happens for Single-Elimination Tournament one-population competitive coevolution.

    prepareForFitnessAssessment will indicate which subpopulations will have their fitness values updated this time around, during postprocessPopulation. It may not be the same as updateFitness[] in evaluate(...).

    If you are basing fitness on trials, this method should create the initial trials if the prepareForFitnessAssessment[...] is true for that subpopulation.

    Specified by:
      `preprocessPopulation` in interface `GroupedProblemForm`

  - ### postprocessPopulation

    public int postprocessPopulation(EvolutionState state, Population pop, boolean[] assessFitness, boolean countVictoriesOnly)
    Description copied from interface: `GroupedProblemForm`
    Finish processing the population (such as fitness information) after evaluation. Although this method is not static, you should not use it to write to any instance variables in the GroupedProblem instance; this is because it's possible that the instance used is in fact the prototype, and you will have no guarantees that your instance variables will remain valid during the evaluate(...) process. Do not assume that `pop` will be the same as `state.pop` -- it may not. `state` is only provided to give you access to EvolutionState features.

    countVictoriesOnly will be set if Individuals' fitness is to be based on whether they're the winner of a test, instead of based on the specifics of the scores in the tests. This really only happens for Single-Elimination Tournament one-population competitive coevolution. If this is set, probably would leave the Fitnesses as they are here (they've been set and incremented in evaluate(...)), but if it's not set, you may want to set the Fitnesses to the maximum or average or the various trials performed.

    assessFitness will indicate which subpopulations should have their final fitness values assessed. You should not clear the trials of individuals for which assessFitness[] is false. Instead allow trials to accumulate and ultimately update the fitnesses later when the flag is set. assessFitness[] may not be the same as updateFitness[] in evaluate(...).

    Should return the number of individuals evaluated (not tested: but actually had their fitnesses modified -- or would have if the evaluated flag wasn't set).

    Specified by:
      `postprocessPopulation` in interface `GroupedProblemForm`

  - ### evaluate

    public void evaluate(EvolutionState state, Individual[] inds, boolean[] updateFitness, boolean countVictoriesOnly, int[] subpops, int threadnum)
    Description copied from interface: `GroupedProblemForm`
    Evaluates the individuals found in ind together. If updateFitness[i] is true, then you should use this evaluation to update the fitness of the individual in ind[i]. Individuals which are updated should have their fitnesses modified so that immediately after evaluation (and prior to postprocessPopulation(...) being called) individuals' fitnesses can be checked to see which is better than which. Do not assume that the individuals in `ind` will actually be in `state.pop` (they may not -- this method may be called at the end of a run to determine the best individual of the run in some kind of contest).

    countVictoriesOnly will be set if Individuals' fitness is to be based on whether they're the winner of a test, instead of based on the specifics of the scores in the tests. This really only happens for Single-Elimination Tournament one-population competitive coevolution. If this is set, you should increment the Fitness of the winner each time. If it's not set, you should update Fitness as you see fit, then set the final Fitness in preprocessPopulation.

    Specified by:
      `evaluate` in interface `GroupedProblemForm`

  - ### initializeContacts

    public void initializeContacts(EvolutionState state)
    Initialize contacts with the slaves
    Overrides:
      `initializeContacts` in class `Problem`

  - ### reinitializeContacts

    public void reinitializeContacts(EvolutionState state)
    Reinitialize contacts with the slaves
    Overrides:
      `reinitializeContacts` in class `Problem`

  - ### closeContacts

    public void closeContacts(EvolutionState state, int result)
    Gracefully close contacts with the slaves
    Overrides:
      `closeContacts` in class `Problem`

  - ### canEvaluate

    public boolean canEvaluate()
    Description copied from class: `Problem`
    Asynchronous Steady-State EC only: Returns true if the problem is ready to evaluate. In most cases, the default is true.
    Overrides:
      `canEvaluate` in class `Problem`

  - ### evaluatedIndividualAvailable

    public boolean evaluatedIndividualAvailable()
    This will only return true if (1) the EvolutionState is a SteadyStateEvolutionState and (2) an individual has returned from the system. If you're not doing steady state evolution, you should not call this method.

  - ### getNextEvaluatedIndividual

    public QueueIndividual getNextEvaluatedIndividual()
    This method blocks until an individual is available from the slaves (which will cause evaluatedIndividualAvailable() to return true), at which time it returns the individual. You should only call this method if you're doing steady state evolution -- otherwise, the method will block forever.

  - ### sendAdditionalData

    public void sendAdditionalData(EvolutionState state, DataOutputStream dataOut)
    This method is called from the SlaveMonitor's accept() thread to optionally send additional data to the Slave via the dataOut stream. By default it does nothing. If you override this you must also override (and use) receiveAdditionalData() and transferAdditionalData().
    Overrides:
      `sendAdditionalData` in class `Problem`

  - ### receiveAdditionalData

    public void receiveAdditionalData(EvolutionState state, DataInputStream dataIn)
    This method is called on a MasterProblem by the Slave. You should use this method to store away received data via the dataIn stream for later transferring to the current EvolutionState via the transferAdditionalData method. You should NOT expect this MasterProblem to be used for by the Slave for evolution (though it might). By default this method does nothing, which is the usual situation. The EvolutionState is provided solely for you to be able to output warnings and errors: do not rely on it for any other purpose (including access of the random number generator or storing any data).
    Overrides:
      `receiveAdditionalData` in class `Problem`

  - ### transferAdditionalData

    public void transferAdditionalData(EvolutionState state)
    This method is called by a Slave to transfer data previously loaded via receiveAdditionalData() to a running EvolutionState at the beginning of evolution. It may be called multiple times if multiple EvolutionStates are created. By default this method does nothing, which is the usual situation.
    Overrides:
      `transferAdditionalData` in class `Problem`

  - ### numComponents

    public int numComponents()
    Specified by:
      `numComponents` in interface `ec.co.ConstructiveProblemForm`
    Returns:
      The total number of components that exist in the problem.

  - ### isCompleteSolution

    public boolean isCompleteSolution(ec.co.ConstructiveIndividual solution)
    Description copied from interface: `ec.co.ConstructiveProblemForm`
    Determine whether a given specifies a complete solution to this problem.
    Specified by:
      `isCompleteSolution` in interface `ec.co.ConstructiveProblemForm`
    Parameters:
      `solution` - A partial or complete solution to this ConstructiveProblemForm.
    Returns:
      True iff the solution is complete (as opposed to partial).

  - ### isViolated

    public boolean isViolated(ec.co.ConstructiveIndividual partialSolution, ec.co.Component component)
    Specified by:
      `isViolated` in interface `ec.co.ConstructiveProblemForm`
    Parameters:
      `partialSolution` - A partial solution to this ConstructiveProblemForm.
      `component` - A component that we may wish to add to the partial solution.
    Returns:
      True iff the constraint allows component to be added to the given partialSolution.

  - ### getAllComponents

    public List<ec.co.Component> getAllComponents()
    Specified by:
      `getAllComponents` in interface `ec.co.ConstructiveProblemForm`
    Returns:
      all components that exist in the problem definition.

  - ### getAllowedComponents

    public List<ec.co.Component> getAllowedComponents(ec.co.ConstructiveIndividual partialSolution)
    Description copied from interface: `ec.co.ConstructiveProblemForm`
    Return the allowable "neighborhood" of components that
    Specified by:
      `getAllowedComponents` in interface `ec.co.ConstructiveProblemForm`
    Parameters:
      `partialSolution` - A collection of components that represent a partial solution that we wish to add a component to.
    Returns:
      The set of all components that can be added to the given partial solution without causing a constraint violation.

  - ### getComponentFromString

    public ec.co.Component getComponentFromString(String s)
    Description copied from interface: `ec.co.ConstructiveProblemForm`
    Decode a String representation of a solution component.
    Specified by:
      `getComponentFromString` in interface `ec.co.ConstructiveProblemForm`
    Parameters:
      `s` - A String representing a component.
    Returns:
      A component matching the provided String.

  - ### getArbitraryComponent

    public ec.co.Component getArbitraryComponent(EvolutionState state, int thread)
    Description copied from interface: `ec.co.ConstructiveProblemForm`
    Chooses an arbitrary component from the problem domain. The intent here is that this gives us an efficient way to choose a component to begin a solution with (as opposed to returning all possible components so that some external method can choose one, which may have nonlinear complexity in some domains).
    Specified by:
      `getArbitraryComponent` in interface `ec.co.ConstructiveProblemForm`
    Parameters:
      `state` - The state (used, for example, to access the simulation's PRNGs)
      `thread` - The thread the caller is operating on. If the caller is single-threaded, just set this to zero.
    Returns:
      An component selected arbitrarily from the problem domain

  - ### isGroupedProblem

    public boolean isGroupedProblem()
    Returns true if the underlying class returns true for isGroupedProblem()
    Overrides:
      `isGroupedProblem` in class `Problem`