Package ec.eval

Class MasterProblem

java.lang.Object
ec.Problem
ec.eval.MasterProblem
All Implemented Interfaces:
ec.co.ConstructiveProblemForm, GroupedProblemForm, Prototype, Setup, SimpleProblemForm, Serializable, Cloneable
public class MasterProblem extends Problem implements SimpleProblemForm, GroupedProblemForm, ec.co.ConstructiveProblemForm
MasterProblem.java

The MasterProblem is a special ECJ problem that performs evaluations by sending them to a remote Slave process to be evaluated. As it implements both the SimpleProblemForm and the GroupedProblemForm interfaces, the MasterProblem can perform both traditional EC evaluations as well as coevolutionary evaluations.

When a MasterProblem is specified by the Evaluator, the Problem is set up as usual, but then the MasterProblem replaces it. The Problem is not garbage collected -- instead, it's hung off the MasterProblem's problem variable. In some sense the Problem is "pushed aside".

If the Evaluator begins by calling prepareToEvaluate(), and we're not doing coevolution, then the MasterProblem does not evaluate individuals immediately. Instead, it waits for at most jobSize individuals be submitted via evaluate(), and then sends them all off in a group, called a job, to the remote slave. In other situations (coevolution, or no prepareToEvaluate()) the MasterProblem sends off individuals immediately.

It may be the case that no Slave has space in its queue to accept a new job containing, among others, your new individual. In this case, calling evaluate() will block until one comes available. You can avoid this by testing for availability first by calling canEvaluate(). Note that canEvaluate() and evaluate() together are not atomic and so you should not rely on this facility if your system uses multiple threads.

When the individuals or their fitnesses return, they are immediately updated in place. You have three options to wait for them:

  • You can wait for all the individuals to finish evaluation by calling finishEvaluating(). If you call this method before a job is entirely filled, it will be sent in truncated format (which generally is perfectly fine). You then block until all the jobs have been completed and the individuals updated.

  • You can block until at least one individual is available, by calling getNextEvaluatedIndividual(), which blocks and then returns the individual that was just completed.

  • You can test in non-blocking fashion to see if an individual is available, by calling evaluatedIndividualAvailable(). If this returns true, you may then call getNextEvaluatedIndividual() to get the individual. Note that this isn't atomic, so don't use it if you have multiple threads.

Parameters

base.debug-info
boolean		 (whether the system should display information useful for debugging purposes)
base.job-size
integer > 0 		(how large should a job be at most?)
eval.master.port
int		 (the port where the slaves will connect)
eval.compression
boolean		 (whether the communication with the slaves should be compressed or not)
eval.masterproblem.max-jobs-per-slave
int		 (the maximum load (number of jobs) per slave at any point in time)
See Also:

  • Field Details

  • Constructor Details

    • MasterProblem

      public MasterProblem()

  • Method Details

    • clone

      public Object clone()
      Description copied from interface: Prototype
      Creates a new individual cloned from a prototype, and suitable to begin use in its own evolutionary context.

      Typically this should be a full "deep" clone. However, you may share certain elements with other objects rather than clone hem, depending on the situation:

      • If you hold objects which are shared with other instances, don't clone them.
      • If you hold objects which must be unique, clone them.
      • If you hold objects which were given to you as a gesture of kindness, and aren't owned by you, you probably shouldn't clone them.
      • DON'T attempt to clone: Singletons, Cliques, or Populations, or Subpopulation.
      • Arrays are not cloned automatically; you may need to clone an array if you're not sharing it with other instances. Arrays have the nice feature of being copyable by calling clone() on them.

      Implementations.

      • If no ancestor of yours implements clone(), and you have no need to do clone deeply, and you are abstract, then you should not declare clone().
      • If no ancestor of yours implements clone(), and you have no need to do clone deeply, and you are not abstract, then you should implement it as follows: 

         public Object clone() 
     {
     try
         { 
         return super.clone();
         }
     catch ((CloneNotSupportedException e)
         { throw new InternalError(); } // never happens
     }
      • If no ancestor of yours implements clone(), but you need to deep-clone some things, then you should implement it as follows: 

         public Object clone() 
     {
     try
         { 
         MyObject myobj = (MyObject) (super.clone());

         // put your deep-cloning code here...
         }
     catch ((CloneNotSupportedException e)
         { throw new InternalError(); } // never happens
     return myobj;
     }
      • If an ancestor has implemented clone(), and you also need to deep clone some things, then you should implement it as follows: 

         public Object clone() 
     { 
     MyObject myobj = (MyObject) (super.clone());

     // put your deep-cloning code here...

     return myobj;
     }
      Specified by:
      clone in interface Prototype
      Overrides:
      clone in class Problem

    • setup

      public void setup(EvolutionState state, Parameter base)
      Description copied from interface: Prototype
      Sets up the object by reading it from the parameters stored in state, built off of the parameter base base. If an ancestor implements this method, be sure to call super.setup(state,base); before you do anything else.

      For prototypes, setup(...) is typically called once for the prototype instance; cloned instances do not receive the setup(...) call. setup(...) may be called more than once; the only guarantee is that it will get called at least once on an instance or some "parent" object from which it was ultimately cloned.

      Specified by:
      setup in interface Prototype
      Specified by:
      setup in interface Setup
      Overrides:
      setup in class Problem

    • prepareToEvaluate

      public void prepareToEvaluate(EvolutionState state, int threadnum)
      Description copied from class: Problem
      May be called by the Evaluator prior to a series of individuals to evaluate, and then ended with a finishEvaluating(...). If this is the case then the Problem is free to delay modifying the individuals or their fitnesses until at finishEvaluating(...). If no prepareToEvaluate(...) is called prior to evaluation, the Problem must complete its modification of the individuals and their fitnesses as they are evaluated as stipulated in the relevant evaluate(...) documentation for SimpleProblemForm or GroupedProblemForm. The default method does nothing. Note that prepareToEvaluate() can be called *multiple times* prior to finishEvaluating() being called -- in this case, the subsequent calls may be ignored.
      Overrides:
      prepareToEvaluate in class Problem

    • finishEvaluating

      public void finishEvaluating(EvolutionState state, int threadnum)
      Description copied from class: Problem
      Will be called by the Evaluator after prepareToEvaluate(...) is called and then a series of individuals are evaluated. However individuals may be evaluated without prepareToEvaluate or finishEvaluating being called at all. See the documentation for prepareToEvaluate for more information. The default method does nothing.
      Overrides:
      finishEvaluating in class Problem

    • evaluate

      public void evaluate(EvolutionState state, Individual ind, int subpopulation, int threadnum)
      Description copied from interface: SimpleProblemForm
      Evaluates the individual in ind, if necessary (perhaps not evaluating them if their evaluated flags are true), and sets their fitness appropriately.
      Specified by:
      evaluate in interface SimpleProblemForm

    • describe

      public void describe(EvolutionState state, Individual ind, int subpopulation, int threadnum, int log)
      Description copied from class: Problem
      Part of SimpleProblemForm. Included here so you don't have to write the default version, which usually does nothing.
      Specified by:
      describe in interface SimpleProblemForm
      Overrides:
      describe in class Problem

    • preprocessPopulation

      public void preprocessPopulation(EvolutionState state, Population pop, boolean[] prepareForFitnessAssessment, boolean countVictoriesOnly)
      Description copied from interface: GroupedProblemForm
      Set up the population pop (such as fitness information) prior to evaluation. Although this method is not static, you should not use it to write to any instance variables in the GroupedProblem instance; this is because it's possible that the instance used is in fact the prototype, and you will have no guarantees that your instance variables will remain valid during the evaluate(...) process. Do not assume that pop will be the same as state.pop -- it may not. state is only provided to give you access to EvolutionState features. Typically you'd use this method to set the Fitness values of all Individuals to 0.

      countVictoriesOnly will be set if Individuals' fitness is to be based on whether they're the winner of a test, instead of based on the specifics of the scores in the tests. This really only happens for Single-Elimination Tournament one-population competitive coevolution.

      prepareForFitnessAssessment will indicate which subpopulations will have their fitness values updated this time around, during postprocessPopulation. It may not be the same as updateFitness[] in evaluate(...).

      If you are basing fitness on trials, this method should create the initial trials if the prepareForFitnessAssessment[...] is true for that subpopulation.

      Specified by:
      preprocessPopulation in interface GroupedProblemForm

    • postprocessPopulation

      public int postprocessPopulation(EvolutionState state, Population pop, boolean[] assessFitness, boolean countVictoriesOnly)
      Description copied from interface: GroupedProblemForm
      Finish processing the population (such as fitness information) after evaluation. Although this method is not static, you should not use it to write to any instance variables in the GroupedProblem instance; this is because it's possible that the instance used is in fact the prototype, and you will have no guarantees that your instance variables will remain valid during the evaluate(...) process. Do not assume that pop will be the same as state.pop -- it may not. state is only provided to give you access to EvolutionState features.

      countVictoriesOnly will be set if Individuals' fitness is to be based on whether they're the winner of a test, instead of based on the specifics of the scores in the tests. This really only happens for Single-Elimination Tournament one-population competitive coevolution. If this is set, probably would leave the Fitnesses as they are here (they've been set and incremented in evaluate(...)), but if it's not set, you may want to set the Fitnesses to the maximum or average or the various trials performed.

      assessFitness will indicate which subpopulations should have their final fitness values assessed. You should not clear the trials of individuals for which assessFitness[] is false. Instead allow trials to accumulate and ultimately update the fitnesses later when the flag is set. assessFitness[] may not be the same as updateFitness[] in evaluate(...).

      Should return the number of individuals evaluated (not tested: but actually had their fitnesses modified -- or would have if the evaluated flag wasn't set).

      Specified by:
      postprocessPopulation in interface GroupedProblemForm

    • evaluate

      public void evaluate(EvolutionState state, Individual[] inds, boolean[] updateFitness, boolean countVictoriesOnly, int[] subpops, int threadnum)
      Description copied from interface: GroupedProblemForm
      Evaluates the individuals found in ind together. If updateFitness[i] is true, then you should use this evaluation to update the fitness of the individual in ind[i]. Individuals which are updated should have their fitnesses modified so that immediately after evaluation (and prior to postprocessPopulation(...) being called) individuals' fitnesses can be checked to see which is better than which. Do not assume that the individuals in ind will actually be in state.pop (they may not -- this method may be called at the end of a run to determine the best individual of the run in some kind of contest).

      countVictoriesOnly will be set if Individuals' fitness is to be based on whether they're the winner of a test, instead of based on the specifics of the scores in the tests. This really only happens for Single-Elimination Tournament one-population competitive coevolution. If this is set, you should increment the Fitness of the winner each time. If it's not set, you should update Fitness as you see fit, then set the final Fitness in preprocessPopulation.

      Specified by:
      evaluate in interface GroupedProblemForm

    • initializeContacts

      public void initializeContacts(EvolutionState state)
      Initialize contacts with the slaves
      Overrides:
      initializeContacts in class Problem

    • reinitializeContacts

      public void reinitializeContacts(EvolutionState state)
      Reinitialize contacts with the slaves
      Overrides:
      reinitializeContacts in class Problem

    • closeContacts

      public void closeContacts(EvolutionState state, int result)
      Gracefully close contacts with the slaves
      Overrides:
      closeContacts in class Problem

    • canEvaluate

      public boolean canEvaluate()
      Description copied from class: Problem
      Asynchronous Steady-State EC only: Returns true if the problem is ready to evaluate. In most cases, the default is true.
      Overrides:
      canEvaluate in class Problem

    • evaluatedIndividualAvailable

      public boolean evaluatedIndividualAvailable()
      This will only return true if (1) the EvolutionState is a SteadyStateEvolutionState and (2) an individual has returned from the system. If you're not doing steady state evolution, you should not call this method.

    • getNextEvaluatedIndividual

      public QueueIndividual getNextEvaluatedIndividual()
      This method blocks until an individual is available from the slaves (which will cause evaluatedIndividualAvailable() to return true), at which time it returns the individual. You should only call this method if you're doing steady state evolution -- otherwise, the method will block forever.

    • sendAdditionalData

      public void sendAdditionalData(EvolutionState state, DataOutputStream dataOut)
      This method is called from the SlaveMonitor's accept() thread to optionally send additional data to the Slave via the dataOut stream. By default it does nothing. If you override this you must also override (and use) receiveAdditionalData() and transferAdditionalData().
      Overrides:
      sendAdditionalData in class Problem

    • receiveAdditionalData

      public void receiveAdditionalData(EvolutionState state, DataInputStream dataIn)
      This method is called on a MasterProblem by the Slave. You should use this method to store away received data via the dataIn stream for later transferring to the current EvolutionState via the transferAdditionalData method. You should NOT expect this MasterProblem to be used for by the Slave for evolution (though it might). By default this method does nothing, which is the usual situation. The EvolutionState is provided solely for you to be able to output warnings and errors: do not rely on it for any other purpose (including access of the random number generator or storing any data).
      Overrides:
      receiveAdditionalData in class Problem

    • transferAdditionalData

      public void transferAdditionalData(EvolutionState state)
      This method is called by a Slave to transfer data previously loaded via receiveAdditionalData() to a running EvolutionState at the beginning of evolution. It may be called multiple times if multiple EvolutionStates are created. By default this method does nothing, which is the usual situation.
      Overrides:
      transferAdditionalData in class Problem

    • numComponents

      public int numComponents()
      Specified by:
      numComponents in interface ec.co.ConstructiveProblemForm
      Returns:
      The total number of components that exist in the problem.

    • isCompleteSolution

      public boolean isCompleteSolution(ec.co.ConstructiveIndividual solution)
      Description copied from interface: ec.co.ConstructiveProblemForm
      Determine whether a given specifies a complete solution to this problem.
      Specified by:
      isCompleteSolution in interface ec.co.ConstructiveProblemForm
      Parameters:
      solution - A partial or complete solution to this ConstructiveProblemForm.
      Returns:
      True iff the solution is complete (as opposed to partial).

    • isViolated

      public boolean isViolated(ec.co.ConstructiveIndividual partialSolution, ec.co.Component component)
      Specified by:
      isViolated in interface ec.co.ConstructiveProblemForm
      Parameters:
      partialSolution - A partial solution to this ConstructiveProblemForm.
      component - A component that we may wish to add to the partial solution.
      Returns:
      True iff the constraint allows component to be added to the given partialSolution.

    • getAllComponents

      public List<ec.co.Component> getAllComponents()
      Specified by:
      getAllComponents in interface ec.co.ConstructiveProblemForm
      Returns:
      all components that exist in the problem definition.

    • getAllowedComponents

      public List<ec.co.Component> getAllowedComponents(ec.co.ConstructiveIndividual partialSolution)
      Description copied from interface: ec.co.ConstructiveProblemForm
      Return the allowable "neighborhood" of components that
      Specified by:
      getAllowedComponents in interface ec.co.ConstructiveProblemForm
      Parameters:
      partialSolution - A collection of components that represent a partial solution that we wish to add a component to.
      Returns:
      The set of all components that can be added to the given partial solution without causing a constraint violation.

    • getComponentFromString

      public ec.co.Component getComponentFromString(String s)
      Description copied from interface: ec.co.ConstructiveProblemForm
      Decode a String representation of a solution component.
      Specified by:
      getComponentFromString in interface ec.co.ConstructiveProblemForm
      Parameters:
      s - A String representing a component.
      Returns:
      A component matching the provided String.

    • getArbitraryComponent

      public ec.co.Component getArbitraryComponent(EvolutionState state, int thread)
      Description copied from interface: ec.co.ConstructiveProblemForm
      Chooses an arbitrary component from the problem domain. The intent here is that this gives us an efficient way to choose a component to begin a solution with (as opposed to returning all possible components so that some external method can choose one, which may have nonlinear complexity in some domains).
      Specified by:
      getArbitraryComponent in interface ec.co.ConstructiveProblemForm
      Parameters:
      state - The state (used, for example, to access the simulation's PRNGs)
      thread - The thread the caller is operating on. If the caller is single-threaded, just set this to zero.
      Returns:
      An component selected arbitrarily from the problem domain

    • isGroupedProblem

      public boolean isGroupedProblem()
      Returns true if the underlying class returns true for isGroupedProblem()
      Overrides:
      isGroupedProblem in class Problem