Get Executor! (Alt Link)
LINK --->>> https://tlniurl.com/2tCJLF
The REST API exposes the values of the Task Metrics collected by Spark executors with the granularityof task execution. The metrics can be used for performance troubleshooting and workload characterization.A list of the available metrics, with a short description:
Executor-level metrics are sent from each executor to the driver as part of the Heartbeat to describe the performance metrics of Executor itself like JVM heap memory, GC information.Executor metric values and their measured memory peak values per executor are exposed via the REST API in JSON format and in Prometheus format.The JSON end point is exposed at: /applications/[app-id]/executors, and the Prometheus endpoint at: /metrics/executors/prometheus.The Prometheus endpoint is conditional to a configuration parameter: spark.ui.prometheus.enabled=true (the default is false).In addition, aggregated per-stage peak values of the executor memory metrics are written to the event log ifspark.eventLog.logStageExecutorMetrics is true.Executor memory metrics are also exposed via the Spark metrics system based on the Dropwizard metrics library.A list of the available metrics, with a short description:
Spark has a configurable metrics system based on theDropwizard Metrics Library.This allows users to report Spark metrics to a variety of sinks including HTTP, JMX, and CSVfiles. The metrics are generated by sources embedded in the Spark code base. Theyprovide instrumentation for specific activities and Spark components.The metrics system is configured via a configuration file that Spark expects to be presentat $SPARK_HOME/conf/metrics.properties. A custom file location can be specified via thespark.metrics.conf configuration property.Instead of using the configuration file, a set of configuration parameters with prefixspark.metrics.conf. can be used.By default, the root namespace used for driver or executor metrics is the value of spark.app.id. However, often times, users want to be able to track the metrics across apps for driver and executors, which is hard to do with application ID (i.e. spark.app.id) since it changes with every invocation of the app. For such use cases,a custom namespace can be specified for metrics reporting using spark.metrics.namespaceconfiguration property. If, say, users wanted to set the metrics namespace to the name of the application, theycan set the spark.metrics.namespace property to a value like ${spark.app.name}. This value isthen expanded appropriately by Spark and is used as the root namespace of the metrics system. Non-driver and executor metrics are never prefixed with spark.app.id, nor does thespark.metrics.namespace property have any such affect on such metrics.
Spring 2.0 introduces a new abstraction for dealing with executors. Executors are the Java 5 name for the concept of thread pools. The \"executor\" naming is due to the fact that there is no guarantee that the underlying implementation is actually a pool; an executor may be single-threaded or even synchronous. Spring's abstraction hides implementation details between Java SE 1.4, Java SE 5 and Java EE environments.
Notice with the above XML that an executor reference is provided for handling those tasks that correspond to methods with the @Async annotation, and the scheduler reference is provided for managing those methods annotated with @Scheduled.
By default when specifying @Async on a method, the executor that will be used is the one supplied to the 'annotation-driven' element as described above. However, the value attribute of the @Async annotation can be used when needing to indicate that an executor other than the default should be used when executing a given method.
As with the scheduler above, the value provided for the 'id' attribute will be used as the prefix for thread names within the pool. As far as the pool size is concerned, the 'executor' element supports more configuration options than the 'scheduler' element. For one thing, the thread pool for a ThreadPoolTaskExecutor is itself more configurable. Rather than just a single size, an executor's thread pool may have different values for the core and the max size. If a single value is provided then the executor will have a fixed-size thread pool (the core and max sizes are the same). However, the 'executor' element's 'pool-size' attribute also accepts a range in the form of \"min-max\".
As you can see from that configuration, a 'queue-capacity' value has also been provided. The configuration of the thread pool should also be considered in light of the executor's queue capacity. For the full description of the relationship between pool size and queue capacity, consult the documentation for ThreadPoolExecutor. The main idea is that when a task is submitted, the executor will first try to use a free thread if the number of active threads is currently less than the core size. If the core size has been reached, then the task will be added to the queue as long as its capacity has not yet been reached. Only then, if the queue's capacity has been reached, will the executor create a new thread beyond the core size. If the max size has also been reached, then the executor will reject the task.
By default, the queue is unbounded, but this is rarely the desired configuration, because it can lead to OutOfMemoryErrors if enough tasks are added to that queue while all pool threads are busy. Furthermore, if the queue is unbounded, then the max size has no effect at all. Since the executor will always try the queue before creating a new thread beyond the core size, a queue must have a finite capacity for the thread pool to grow beyond the core size (this is why a fixed size pool is the only sensible case when using an unbounded queue).
In a moment, we will review the effects of the keep-alive setting which adds yet another factor to consider when providing a pool size configuration. First, let's consider the case, as mentioned above, when a task is rejected. By default, when a task is rejected, a thread pool executor will throw a TaskRejectedException. However, the rejection policy is actually configurable. The exception is thrown when using the default rejection policy which is the AbortPolicy implementation. For applications where some tasks can be skipped under heavy load, either the DiscardPolicy or DiscardOldestPolicy may be configured instead. Another option that works well for applications that need to throttle the submitted tasks under heavy load is the CallerRunsPolicy. Instead of throwing an exception or discarding tasks, that policy will simply force the thread that is calling the submit method to run the task itself. The idea is that such a caller will be busy while running that task and not able to submit other tasks immediately. Therefore it provides a simple way to throttle the incoming load while maintaining the limits of the thread pool and queue. Typically this allows the executor to \"catch up\" on the tasks it is handling and thereby frees up some capacity on the queue, in the pool, or both. Any of these options can be chosen from an enumeration of values available for the 'rejection-policy' attribute on the 'executor' element.
Glancing through the list of the executor's duties should tip you off about who might be the best person for the job. The prime characteristics are honesty, skill at organizing and finesse in keeping track of details. For many tasks, such as collecting mail and finding important records and papers, it may be most helpful to name someone who lives nearby or who is familiar with your business matters.
Following are a few more things you may want to consider when naming your executor for Nolo's Online Will. Whomever you choose, make sure the person you select is willing to do the job. Discuss the possible duties involved with your choice for executor before naming him or her in your will.
As a practical matter, it's wise to name an executor who lives close to you. It will be more difficult for the executor to handle day-to-day matters from a distance. But if the best person for the job lives far away, there's no law against naming that person in your will. Every state allows out of-state executors to serve, though many states will require a bond and most states impose special rules on out-of-state executors.
If you do not name an executor in your will, the document will still be valid. But your decision will not have been a wise one. It will most often mean that a court will have to scurry to come up with a willing relative to serve. If that fails, the court will probably appoint someone to do the job who is likely to be unfamiliar with you, your property and your beneficiaries. People appointed by the court to serve are usually called administrators.
The laws in many states provide that anyone who is entitled under the will to take over half a person's property has first priority to serve as executor. If no such person is apparent, courts will generally look for someone to serve among the following groups of people, in the following order:
EXAMPLE: Marsha names Bill and Jane as her coexecutors. She then names Susan as first alternate and Keith as second alternate. When Marsha dies, if either Bill or Jane is unavailable to be the executor of her estate, the other will serve alone. If both Bill and Jane are unavailable, Susan will serve. In the unlikely event that Bill, Jane and Susan are all unavailable, it will be up to Keith to wrap up Marsha's estate.
Selecting an executor from among those who will inherit property under the will is common. Self-interest should ensure that they will protect assets, maintain the real estate and ensure the timely completion of the probate process.
However, you may not have surviving family members or friends or do not want them to serve as your executor. You can hire a professional executor to administer your estate, such as a bank, a trust company, a CPA, a corporate trustee, or a probate law firm.
Yes, however, it may complicate things. Often so