Academia.eduAcademia.edu

Figure 75 – uploaded by Kyle Woerner

See full PDFdownloadDownload figure
Figure D-1 This still image represents the most meaningful part of Scenario 1 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.

Figure 75 D-1 This still image represents the most meaningful part of Scenario 1 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.

Related Figures (132)

5 Performance Testing and Simulation Methods
5 Performance Testing and Simulation Methods
A COLREGS Reference
A COLREGS Reference
Figure 2-1 Larson showed that the COLREGS collision avoidance problem was dis- tinctly reactive in [47]. Image is Figure 3 from [47].
Figure 2-1 Larson showed that the COLREGS collision avoidance problem was dis- tinctly reactive in [47]. Image is Figure 3 from [47].
Figure 2-2. Tam introduced varying safety areas based on ownship and obstacle parameters using a fore and aft half-elliptical model. Image is Figure 6 from [77]. n vehicle shape and other uncertainties. Tam’s original graphic from [77] is showr Kim et al. [43] introduce the concept of Minimum Range Requirement to account
Figure 2-2. Tam introduced varying safety areas based on ownship and obstacle parameters using a fore and aft half-elliptical model. Image is Figure 6 from [77]. n vehicle shape and other uncertainties. Tam’s original graphic from [77] is showr Kim et al. [43] introduce the concept of Minimum Range Requirement to account
the literature.
the literature.
‘igure 2-4 A nominal velocity obstacle utility function is shown representing the stepwise increase in safety as a resulting maneuver’s range crosses the a priori determined collision range. function representing the maneuvers determined to be safe for a contact.
‘igure 2-4 A nominal velocity obstacle utility function is shown representing the stepwise increase in safety as a resulting maneuver’s range crosses the a priori determined collision range. function representing the maneuvers determined to be safe for a contact.
Figure 2-5 The aggregate of utility values for all candidate course-speed pairs is shown in a polar heat map where red is desirable (safe) and blue is undesirable (unsafe).
Figure 2-5 The aggregate of utility values for all candidate course-speed pairs is shown in a polar heat map where red is desirable (safe) and blue is undesirable (unsafe).
Figure 2-7 Kuwata applied the traditional velocity obstacle with an additional ex- clusion range for maneuvers that were said to be non-compliant with COLREGS Image is Figure 10 from [45]. detected using onboard sensors.
Figure 2-7 Kuwata applied the traditional velocity obstacle with an additional ex- clusion range for maneuvers that were said to be non-compliant with COLREGS Image is Figure 10 from [45]. detected using onboard sensors.
Figure 2-8 Choi used a safety area based on fixed range of ownship (airplane). Image is Figure 2 from [14]. constraints were not invoked. esults. A simulation of up to 8 simultaneous simulated aircraft demonstrated a star- Benjamin et al. first demonstrated on-water experimentation with CPA-based
Figure 2-8 Choi used a safety area based on fixed range of ownship (airplane). Image is Figure 2 from [14]. constraints were not invoked. esults. A simulation of up to 8 simultaneous simulated aircraft demonstrated a star- Benjamin et al. first demonstrated on-water experimentation with CPA-based
t number of vehicles displayed as autonomous + human-operated Table 2.1 Summary of Literature Attributes Relative to this Thesis
t number of vehicles displayed as autonomous + human-operated Table 2.1 Summary of Literature Attributes Relative to this Thesis
Autonomous surface vessels require course and speed input resulting from a weightec both of these variables. For other physical domains, altitude or depth decisions are
Autonomous surface vessels require course and speed input resulting from a weightec both of these variables. For other physical domains, altitude or depth decisions are
Figure 3-2 The velocity obstacle function of Equation (3.23) is shown as a step function at the velocity obstacle’s collision range (RY).
Figure 3-2 The velocity obstacle function of Equation (3.23) is shown as a step function at the velocity obstacle’s collision range (RY).
igure 3-4 In (a), a nominal CPA utility function shows the utility for a single threshold mapping. In the case of a range-based function, the resulting maneu- ver’s range crosses the a priori determined “safe” range for a single range threshold and the velocity obstacle is recovered. In (b-c), a two-threshold, three-component utility function represents the collision avoidance objective function. Arbitrary in- terior functions can be used for a more advanced utility mapping than possible by velocity obstacles. The three-component linear function of (c) represents Equa- tion (3.25) with an independent variable of range at CPA. In (d), an arbitrary non-uniform utility function is shown. The discriminators 9 allow selection of the appropriate component function f; based on one or more conditions usually involv- ing range, time, pose, and collision avoidance rule. Expansion to more generalized collision avoidance utility functions allows for greater reasoning within the larger autonomous vessel mission scheme. Independent variables other than range can be designed, and multiple discriminators can be used. An example second discrimina- tor would be to map all tepa > thorizon to the fj = 1 component function without further evaluation of range.
igure 3-4 In (a), a nominal CPA utility function shows the utility for a single threshold mapping. In the case of a range-based function, the resulting maneu- ver’s range crosses the a priori determined “safe” range for a single range threshold and the velocity obstacle is recovered. In (b-c), a two-threshold, three-component utility function represents the collision avoidance objective function. Arbitrary in- terior functions can be used for a more advanced utility mapping than possible by velocity obstacles. The three-component linear function of (c) represents Equa- tion (3.25) with an independent variable of range at CPA. In (d), an arbitrary non-uniform utility function is shown. The discriminators 9 allow selection of the appropriate component function f; based on one or more conditions usually involv- ing range, time, pose, and collision avoidance rule. Expansion to more generalized collision avoidance utility functions allows for greater reasoning within the larger autonomous vessel mission scheme. Independent variables other than range can be designed, and multiple discriminators can be used. An example second discrimina- tor would be to map all tepa > thorizon to the fj = 1 component function without further evaluation of range.
Figure 3-5 The selection of priority weight allows multiple objective functions to be added in accordance with their relative importance to select the globally opti- mal solution. Shown here are four priority weight functions. Each priority weight function is piecewise defined over some domain of its independent variable (e.g., instantaneous range, current time to CPA, etc.). The conditions that select the appropriate piece of the function to be used as the priority weight function are its discriminators (). In (a), a nominal priority weight function shows the priority for a single threshold mapping. The resulting maneuver’s independent variable (range) crosses the a priori determined range at which collision avoidance maneuvers be- come important for a single range threshold. In (b-c), a dual-range threshold is used and a linear or quadratic function can be used for a more dynamic priority weight mapping. In (d), an arbitrary function is used to determine priority weight.
Figure 3-5 The selection of priority weight allows multiple objective functions to be added in accordance with their relative importance to select the globally opti- mal solution. Shown here are four priority weight functions. Each priority weight function is piecewise defined over some domain of its independent variable (e.g., instantaneous range, current time to CPA, etc.). The conditions that select the appropriate piece of the function to be used as the priority weight function are its discriminators (). In (a), a nominal priority weight function shows the priority for a single threshold mapping. The resulting maneuver’s independent variable (range) crosses the a priori determined range at which collision avoidance maneuvers be- come important for a single range threshold. In (b-c), a dual-range threshold is used and a linear or quadratic function can be used for a more dynamic priority weight mapping. In (d), an arbitrary function is used to determine priority weight.
Algorithm 1 Full CPA quantification algorithm for determining CPA range, time, and pose for sampled candidate maneuvers under COLREGS constraints. such that AM OBS;. Using Figure 3-3, A € R%% implies a CPA range less than R?%,
Algorithm 1 Full CPA quantification algorithm for determining CPA range, time, and pose for sampled candidate maneuvers under COLREGS constraints. such that AM OBS;. Using Figure 3-3, A € R%% implies a CPA range less than R?%,
Algorithm 2 Full CPA algorithm for determining collision avoidance utility for sam- pled candidate maneuvers under COLREGS constraints using range-only discrimina- tion of Equation (3.26). regardless of whether the contact is human operated, autonomous, or some combina- tion thereof. To see the benefits of considering additional calculations to garner explicit collision
Algorithm 2 Full CPA algorithm for determining collision avoidance utility for sam- pled candidate maneuvers under COLREGS constraints using range-only discrimina- tion of Equation (3.26). regardless of whether the contact is human operated, autonomous, or some combina- tion thereof. To see the benefits of considering additional calculations to garner explicit collision
Figure 3-7 The full CPA quantification algorithm of Equation (3.27) is shown as an arbitrary continuous function for a two-threshold range configuration. the circumstances of the case admit. This limitation significantly reduces the decision
Figure 3-7 The full CPA quantification algorithm of Equation (3.27) is shown as an arbitrary continuous function for a two-threshold range configuration. the circumstances of the case admit. This limitation significantly reduces the decision
Figure 3-8 This polar objective function with radius = speed (v), polar angle (north up) = course (6, North-up) depicts raw CPA ranges varying from smallest (dark blue) to largest (dark red). The red plateau depicts maneuvers where an opening aspect would occur, i.e., CPA is in the past. subset of decision space with a means of inferring the risk involved.
Figure 3-8 This polar objective function with radius = speed (v), polar angle (north up) = course (6, North-up) depicts raw CPA ranges varying from smallest (dark blue) to largest (dark red). The red plateau depicts maneuvers where an opening aspect would occur, i.e., CPA is in the past. subset of decision space with a means of inferring the risk involved.
‘igure 3-9 These polar objective functions with radius = speed (v), polar angle (north up) = course (6, North-up) represent the 2’(0,v) decision space for the sin- gle vehicle-pair co algorithm of Figure 3-3. T utility = 1) ande lision avoidance scenario of Figure 3-1 using a velocity obstacle he velocity obstacle algorithm shows both allowed (red, iminated (blue, utility = 0) candidate maneuvers without (Fig- ure 3-9a) and with (Figure 3-9b) COLREGS protocol constraints (give-way vessel crossing astern of the stand-on vessel). The stepwise cliff in utility occurs along the lines corresponding to @ crossing into the excluded region of the velocity obstacle or violating a COLRI EGS pro tocol constraint. The amount of allowed (red) candidate maneuvers can be significantly reduced when protocol constraints are applied.
‘igure 3-9 These polar objective functions with radius = speed (v), polar angle (north up) = course (6, North-up) represent the 2’(0,v) decision space for the sin- gle vehicle-pair co algorithm of Figure 3-3. T utility = 1) ande lision avoidance scenario of Figure 3-1 using a velocity obstacle he velocity obstacle algorithm shows both allowed (red, iminated (blue, utility = 0) candidate maneuvers without (Fig- ure 3-9a) and with (Figure 3-9b) COLREGS protocol constraints (give-way vessel crossing astern of the stand-on vessel). The stepwise cliff in utility occurs along the lines corresponding to @ crossing into the excluded region of the velocity obstacle or violating a COLRI EGS pro tocol constraint. The amount of allowed (red) candidate maneuvers can be significantly reduced when protocol constraints are applied.
igure 3-10 These polar objective functions showing the same collision avoidance scenario of Figure 3-1 demonstrate the continuous utility values available when us- ing the full CPA quantification demonstrated in Figure 3-6 and Algorithm 1. Both allowed (red, utility = 1) and eliminated (blue, utility = 0) candidate maneuvers of this figure are consistent with the velocity obstacle. Intermediate continuous utili- ties (other colors) are allowed based on direct computation of range at CPA both without (Figure 3-10a) and with (Figure 3-10b) the added COLREGS protocol con- straints for a give-way vessel crossing astern of the stand-on vessel (Rules 15/16). With the significant reduction in feasible candidate maneuvers after applying the COLREGS rules, the full CPA quantification algorithm allows direct sampling of otherwise excluded candidate maneuvers. Those sampled candidate maneuvers de- termined to have a resulting range at CPA (repa) between the minimum allowed (R,nin) and preferred (R,,.¢) ranges at CPA are considered for selection with a utility value commensurate with their corresponding value of Tepq.
igure 3-10 These polar objective functions showing the same collision avoidance scenario of Figure 3-1 demonstrate the continuous utility values available when us- ing the full CPA quantification demonstrated in Figure 3-6 and Algorithm 1. Both allowed (red, utility = 1) and eliminated (blue, utility = 0) candidate maneuvers of this figure are consistent with the velocity obstacle. Intermediate continuous utili- ties (other colors) are allowed based on direct computation of range at CPA both without (Figure 3-10a) and with (Figure 3-10b) the added COLREGS protocol con- straints for a give-way vessel crossing astern of the stand-on vessel (Rules 15/16). With the significant reduction in feasible candidate maneuvers after applying the COLREGS rules, the full CPA quantification algorithm allows direct sampling of otherwise excluded candidate maneuvers. Those sampled candidate maneuvers de- termined to have a resulting range at CPA (repa) between the minimum allowed (R,nin) and preferred (R,,.¢) ranges at CPA are considered for selection with a utility value commensurate with their corresponding value of Tepq.
Figure 3-13 An arbitrary collision avoidance objective function is shown with IvP functions sampling at uniform sampling size and shape without refinement of any kind; here, red represents a high utility while blue represents an undesirable utility. This worst case sampling for the given number of pieces shows that a reasonably small sample size provides an accurate representation of the underling objective function. By using IvP rather than a simple exclusionary technique such as the velocity obstacles family of algorithms, allowed regions of course-speed candidates can be incorporated into the selection of the globally optimal velocity vector. IvP approximations of the underlying objective function (e) are shown using 64, 500, 1000, and 4000 uniformly spaced and sized IvP pieces.
Figure 3-13 An arbitrary collision avoidance objective function is shown with IvP functions sampling at uniform sampling size and shape without refinement of any kind; here, red represents a high utility while blue represents an undesirable utility. This worst case sampling for the given number of pieces shows that a reasonably small sample size provides an accurate representation of the underling objective function. By using IvP rather than a simple exclusionary technique such as the velocity obstacles family of algorithms, allowed regions of course-speed candidates can be incorporated into the selection of the globally optimal velocity vector. IvP approximations of the underlying objective function (e) are shown using 64, 500, 1000, and 4000 uniformly spaced and sized IvP pieces.
‘igure 3-16 An arbitrary complex collision avoidance objective function without ap- proximation (a) demonstrates the complexities of multi-contact collision avoidance. Red colors again represent desirable conditions while blues represent unfavorable collision conditions (ranges less than Rin, or equivalently, excluded regions from multi-contact velocity obstacle solutions). The topographic view (b) demonstrates the existence of three highly desired red peaks. By again bisecting the true ob- jective function with a plane at z =Rypjn, the black plane represents the exclusion zone of velocity obstacles (c). By choosing some z >Rmmin, the more highly desirable solutions with ranges at CPA approaching Rpre¢ become evident to present a more decision-rich input to the velocity vector solution.
‘igure 3-16 An arbitrary complex collision avoidance objective function without ap- proximation (a) demonstrates the complexities of multi-contact collision avoidance. Red colors again represent desirable conditions while blues represent unfavorable collision conditions (ranges less than Rin, or equivalently, excluded regions from multi-contact velocity obstacle solutions). The topographic view (b) demonstrates the existence of three highly desired red peaks. By again bisecting the true ob- jective function with a plane at z =Rypjn, the black plane represents the exclusion zone of velocity obstacles (c). By choosing some z >Rmmin, the more highly desirable solutions with ranges at CPA approaching Rpre¢ become evident to present a more decision-rich input to the velocity vector solution.
‘igure 3-17 These polar heat maps show course (angle) and speed (radius) for pa- tience parameters with values of 7g = 30 (a) and a = 70 (b). Desirable velocities are dark red while less desirable velocities are blue. Both examples are trying to achieve a course of 180°. The fuchsia point represents the peak value of each objective.
‘igure 3-17 These polar heat maps show course (angle) and speed (radius) for pa- tience parameters with values of 7g = 30 (a) and a = 70 (b). Desirable velocities are dark red while less desirable velocities are blue. Both examples are trying to achieve a course of 180°. The fuchsia point represents the peak value of each objective.
accompany certain primary mission characteristics. For example, a vessel may be
accompany certain primary mission characteristics. For example, a vessel may be
able 4.1 Categories of Scope for Evaluation of COLREGS Compliance and response) to other authors. By defining the scope of applicable rules and demon-
able 4.1 Categories of Scope for Evaluation of COLREGS Compliance and response) to other authors. By defining the scope of applicable rules and demon-
function of both range and pose is shown in Algorithm 3.
function of both range and pose is shown in Algorithm 3.
be the objective when integrating autonomous systems into human-present environ-
be the objective when integrating autonomous systems into human-present environ-
Ba Algorithm 5 COLREGS Entry Criteria: Determining the Appropriate Rule Set
Ba Algorithm 5 COLREGS Entry Criteria: Determining the Appropriate Rule Set
igure 4-3 Ownship (labeled “O/S”) is traveling east and first sites a contact at relative bearing 6 with contact angle a in (a). Speed is represented by the length of the colored lines from each vessel (red for contact, blue for ownship). From the perspective of the contact looking at ownship, a and ( are simply interchanged. These two angles give great insight into the collision avoidance picture and quickly aid in determining the applicable protocol constraints. Combined with CPA range and time (Pepa; tepa), pose at CPA (Ocpa = (Qepas Bepa)) gives important information as to risk of collision, collision avoidance protocol compliance, and overall safety of amaneuver. Relative bearing and contact angle at CPA are shown in (b). CPA (b) occurs when the range between contacts reaches its minimum value of the encounter.
igure 4-3 Ownship (labeled “O/S”) is traveling east and first sites a contact at relative bearing 6 with contact angle a in (a). Speed is represented by the length of the colored lines from each vessel (red for contact, blue for ownship). From the perspective of the contact looking at ownship, a and ( are simply interchanged. These two angles give great insight into the collision avoidance picture and quickly aid in determining the applicable protocol constraints. Combined with CPA range and time (Pepa; tepa), pose at CPA (Ocpa = (Qepas Bepa)) gives important information as to risk of collision, collision avoidance protocol compliance, and overall safety of amaneuver. Relative bearing and contact angle at CPA are shown in (b). CPA (b) occurs when the range between contacts reaches its minimum value of the encounter.
Figure 4-6 COLREGS pose reward functions can be used to give preference to pass- ing contacts with relative bearing and contact angle that are least likely to increase risk of collision. This function rewards beam and stern contact angles at CPA using Equation (4.6). This example shows the cutoff angle set to A, = 90° to give equal preference to beam and stern aspects. No reward is given for beam aspects.
Figure 4-6 COLREGS pose reward functions can be used to give preference to pass- ing contacts with relative bearing and contact angle that are least likely to increase risk of collision. This function rewards beam and stern contact angles at CPA using Equation (4.6). This example shows the cutoff angle set to A, = 90° to give equal preference to beam and stern aspects. No reward is given for beam aspects.
‘igure 4-9 Rule 14 requires head-on contacts to maneuver to starboard and pass port-to-port. The geometry of a nearly canonical case (a) shows the preferred CPA geometry including relative bearing @ and contact angle a. Using Equation (4.12), a nearly maximum pose score would result. In (b), a delayed maneuver from ownship (“O/S”) results in a port relative bearing at CPA; contact angle Aepa, however, accounts for the less than ideal CPA geometry. Equation (4.11) would reduce the overall performance score for the situation in (b).
‘igure 4-9 Rule 14 requires head-on contacts to maneuver to starboard and pass port-to-port. The geometry of a nearly canonical case (a) shows the preferred CPA geometry including relative bearing @ and contact angle a. Using Equation (4.12), a nearly maximum pose score would result. In (b), a delayed maneuver from ownship (“O/S”) results in a port relative bearing at CPA; contact angle Aepa, however, accounts for the less than ideal CPA geometry. Equation (4.11) would reduce the overall performance score for the situation in (b).
Algorithm 8 Rule 14: Head-on Vessels
Algorithm 8 Rule 14: Head-on Vessels
the linear region. Several small turns resulting in a larger effective turn should also Algorithm 13 Penalize for Non-Readily Apparent Maneuver
the linear region. Several small turns resulting in a larger effective turn should also Algorithm 13 Penalize for Non-Readily Apparent Maneuver
4.5.2 Responsibilities of Vessels within Sight: Rules 11, 18 Identification of the contact’s type (e.g., power-driven, sailing, etc.) gives necessary
4.5.2 Responsibilities of Vessels within Sight: Rules 11, 18 Identification of the contact’s type (e.g., power-driven, sailing, etc.) gives necessary
Figure 4-13 After separating rules into categories, graphical displays such as this can demonstrate the scope of work as well as the level of compliance within the scope. Here a report shows work in autonomous collision avoidance for Categories I- V with performance below the configurable threshold for “compliance” visually indicated by the red dashed line at a score value of 80%. The evaluation of this example certification authority uses an unweighted average across Categories I- IX to compute a Category X score. This thesis primarily considers evaluation of Categories II,V, and VI.
Figure 4-13 After separating rules into categories, graphical displays such as this can demonstrate the scope of work as well as the level of compliance within the scope. Here a report shows work in autonomous collision avoidance for Categories I- V with performance below the configurable threshold for “compliance” visually indicated by the red dashed line at a score value of 80%. The evaluation of this example certification authority uses an unweighted average across Categories I- IX to compute a Category X score. This thesis primarily considers evaluation of Categories II,V, and VI.
Figure 4-14 Scoring of COLREGS collision avoidance rules allow for real-time eval- uation of vehicle performance at the shoreside observation center. Configurable range parameters include nominal detection range, preferred range at CPA, mini- mum acceptable range at CPA, threshold range at which a near-miss occurs, and the range at which a collision is assumed. An aggregate tally of COLREGS viola- tions (scores below a configurable threshold value) and of each configuration range are displayed. Vehicle types as specified by each vessel are used as a filter to allow consideration of only certain entities within the “visibility” of the shoreside ob- servation center. This allows underwater, ground, and aerial vehicles to share the shoreside observation display without unnecessarily being considered as COLREGS compliance candidates.
Figure 4-14 Scoring of COLREGS collision avoidance rules allow for real-time eval- uation of vehicle performance at the shoreside observation center. Configurable range parameters include nominal detection range, preferred range at CPA, mini- mum acceptable range at CPA, threshold range at which a near-miss occurs, and the range at which a collision is assumed. An aggregate tally of COLREGS viola- tions (scores below a configurable threshold value) and of each configuration range are displayed. Vehicle types as specified by each vessel are used as a filter to allow consideration of only certain entities within the “visibility” of the shoreside ob- servation center. This allows underwater, ground, and aerial vehicles to share the shoreside observation display without unnecessarily being considered as COLREGS compliance candidates.
Figure 4-15 Violations of range below configurable threshold values ((a) minimum desired, (b) near-miss , and (c) collision) result in display of green, yellow, then red rings, respectively, around the vehicle and sounding of an optional audible indicator. This allows real-time warning to an evaluator of a dangerous collision situation. Violations of COLRI EGS collision avoidance rules below a configurable threshold value result in display of a blue ring (d) around the vehicle and sounding of an optional audible indicator.
Figure 4-15 Violations of range below configurable threshold values ((a) minimum desired, (b) near-miss , and (c) collision) result in display of green, yellow, then red rings, respectively, around the vehicle and sounding of an optional audible indicator. This allows real-time warning to an evaluator of a dangerous collision situation. Violations of COLRI EGS collision avoidance rules below a configurable threshold value result in display of a blue ring (d) around the vehicle and sounding of an optional audible indicator.
Figure 4-19 Mean protocol compliance for three protocol constrained vehicles and one collision agnostic vehicle demonstrated the ability to identify the offending vehicle. A total of 1,788 encounters demonstrated that the most clear reduction in compliance score occurred for the head-on case (9*'*). The stand-on rules were largely compliant as maintaining course and speed was natural for the collision agnostic vehicle. A lower than perfect score for 9%'5/"7 resulted from the few cases where the collision agnostic vehicle failed to take evasive action when in extremis.
Figure 4-19 Mean protocol compliance for three protocol constrained vehicles and one collision agnostic vehicle demonstrated the ability to identify the offending vehicle. A total of 1,788 encounters demonstrated that the most clear reduction in compliance score occurred for the head-on case (9*'*). The stand-on rules were largely compliant as maintaining course and speed was natural for the collision agnostic vehicle. A lower than perfect score for 9%'5/"7 resulted from the few cases where the collision agnostic vehicle failed to take evasive action when in extremis.
Table 4.2 Significance Testing for Compliance Detection 4.7.4 Human Agreement with Evaluation Results vehicles by the evaluation algorithms to significance values of p < 0.01. Nine collision avoidance scenarios were analyzed by the evaluation tools presented in
Table 4.2 Significance Testing for Compliance Detection 4.7.4 Human Agreement with Evaluation Results vehicles by the evaluation algorithms to significance values of p < 0.01. Nine collision avoidance scenarios were analyzed by the evaluation tools presented in
Table 4.4 Significance Testing: Degrees of Freedom Table 4.3 Significance Testing: t-Test Values
Table 4.4 Significance Testing: Degrees of Freedom Table 4.3 Significance Testing: t-Test Values
rarely receives more than 25% blame in a collision though often receives some blame
rarely receives more than 25% blame in a collision though often receives some blame
Figure 5-2. A vessel may maneuver in such a way as to be readily apparent but also result in a dangerous and unsafe CPA. The apparent maneuver is necessary but not sufficient to warrant a high compliance score. The egregiousness score assists designers and evaluators in prioritizing collision avoidance encounters for analysis and corrective action
Figure 5-2. A vessel may maneuver in such a way as to be readily apparent but also result in a dangerous and unsafe CPA. The apparent maneuver is necessary but not sufficient to warrant a high compliance score. The egregiousness score assists designers and evaluators in prioritizing collision avoidance encounters for analysis and corrective action
In scenarios of evaluation using proportional blame assignment, blame may be initial blame may be assigned by one of two methods:
In scenarios of evaluation using proportional blame assignment, blame may be initial blame may be assigned by one of two methods:
‘igure 5-4 An example three-vehicle encounter (a) shows the initial positions of a random experiment. The dot in the center represents the simultaneous collision point. Distance from the collision point represents the initial speed of the vehicle where farther distance implies faster initial speed. Slow speed vessels A and B are in nominal crossing patterns with initial heading noise to preclude canonical geometries (b) while high-speed vessel C' assumes initial headings throughout the range of 0° to 360°. Each geometry configuration G is iterated over the variations of the design of experiments.
‘igure 5-4 An example three-vehicle encounter (a) shows the initial positions of a random experiment. The dot in the center represents the simultaneous collision point. Distance from the collision point represents the initial speed of the vehicle where farther distance implies faster initial speed. Slow speed vessels A and B are in nominal crossing patterns with initial heading noise to preclude canonical geometries (b) while high-speed vessel C' assumes initial headings throughout the range of 0° to 360°. Each geometry configuration G is iterated over the variations of the design of experiments.
Table 5.1 Iterative Geometric Testing Attributes encountering two slower speed vessels. 5.5 Case Study on High-Speed Vessels tion 5.5 presents a case study of iterative geometric testing for a high-speed vessel The iterative geometric testing framework was applied to a high-speed vessel encoun-
Table 5.1 Iterative Geometric Testing Attributes encountering two slower speed vessels. 5.5 Case Study on High-Speed Vessels tion 5.5 presents a case study of iterative geometric testing for a high-speed vessel The iterative geometric testing framework was applied to a high-speed vessel encoun-
Table 5.2 Iterative Geometry Experimentation Configuration
Table 5.2 Iterative Geometry Experimentation Configuration
Figure 5-6 Instantaneous relative geometries are shown for comparison of the ob- jective functions of Figure 5-7 (vessel size not to scale). The high-speed vessel is shown in red at a critical decision point of whether to speed around vessel A or slow and allow vessel A to proceed down its track.
Figure 5-6 Instantaneous relative geometries are shown for comparison of the ob- jective functions of Figure 5-7 (vessel size not to scale). The high-speed vessel is shown in red at a critical decision point of whether to speed around vessel A or slow and allow vessel A to proceed down its track.
Figure 5-7 The high-speed vessel’s objective functions for patience parameters of T = 0.30 (left column) and 7» = 0.70 (right column). These polar objective functions represent the high-speed vessel’s decision space. In this heat map of candidate velocity vectors, red is most desired while blue is least desired. Radius corresponds to desired velocity; maximum speed is represented by the outer edge of each circle The nolar anole eorresnonds to heading with North heinoe vertical.
Figure 5-7 The high-speed vessel’s objective functions for patience parameters of T = 0.30 (left column) and 7» = 0.70 (right column). These polar objective functions represent the high-speed vessel’s decision space. In this heat map of candidate velocity vectors, red is most desired while blue is least desired. Radius corresponds to desired velocity; maximum speed is represented by the outer edge of each circle The nolar anole eorresnonds to heading with North heinoe vertical.
Figure 5-8 The efficiencies varied for the high-speed vessel depending on the pa- tience parameter chosen. The spatial and temporal efficiencies for the experiment of Figures 5-6 and 5-7 demonstrates that a significant fluctuation in efficiencies is possible when changing the patience parameter. Careful study of the affects of configuration settings on performance parameters are necessary extending a known configuration to a new type of vessel. Here the experiment performance values are shown in blue. The mean performance over all geometries for the same value of patience parameter is shown in blue.
Figure 5-8 The efficiencies varied for the high-speed vessel depending on the pa- tience parameter chosen. The spatial and temporal efficiencies for the experiment of Figures 5-6 and 5-7 demonstrates that a significant fluctuation in efficiencies is possible when changing the patience parameter. Careful study of the affects of configuration settings on performance parameters are necessary extending a known configuration to a new type of vessel. Here the experiment performance values are shown in blue. The mean performance over all geometries for the same value of patience parameter is shown in blue.
Figure 5-9 The high-speed vehicle performance was compared across the four per- formance metrics of spatial efficiency, temporal efficiency, safety, and protocol com- pliance. By evaluating performance using different metrics, trades can be made based on metrics that are given preference. Note the wide variation of protocol compliance. The high-speed vessel is shown in yellow while slow speed vessels A and B are shown in red and blue, respectively.
Figure 5-9 The high-speed vehicle performance was compared across the four per- formance metrics of spatial efficiency, temporal efficiency, safety, and protocol com- pliance. By evaluating performance using different metrics, trades can be made based on metrics that are given preference. Note the wide variation of protocol compliance. The high-speed vessel is shown in yellow while slow speed vessels A and B are shown in red and blue, respectively.
having sufficiently and significantly populated data to make a tuning decision suck
having sufficiently and significantly populated data to make a tuning decision suck
Figure 5-10 The minimum safety score of each encounter allowed the limiting case in a multi-contact collision avoidance scenario to be identified. Statistics based on this data give insight to performance for either fixed geometry or fixed patience parameter. Course weights shown on the horizontal axis represent 100-7». For a well-populated and statistically significant data set, an analysis using a method such as this chart could give insight into tuning preferences to maximize performance metrics (safety, efficiency, protocol compliance, etc.). This example data indicate that most geometry and patience combinations yield a minimum safety greater than 80; additional testing in areas below 80 may provide interesting edge cases for particular combinations of geometry and patience values. Additional testing would be necessary to draw conclusions worthy of tuning. If an initial contact geometry could be determined and the data populating a chart such as this were sufficient and significant, tuning of experimental variables would be possible to operate according to a designer’s policy.
Figure 5-10 The minimum safety score of each encounter allowed the limiting case in a multi-contact collision avoidance scenario to be identified. Statistics based on this data give insight to performance for either fixed geometry or fixed patience parameter. Course weights shown on the horizontal axis represent 100-7». For a well-populated and statistically significant data set, an analysis using a method such as this chart could give insight into tuning preferences to maximize performance metrics (safety, efficiency, protocol compliance, etc.). This example data indicate that most geometry and patience combinations yield a minimum safety greater than 80; additional testing in areas below 80 may provide interesting edge cases for particular combinations of geometry and patience values. Additional testing would be necessary to draw conclusions worthy of tuning. If an initial contact geometry could be determined and the data populating a chart such as this were sufficient and significant, tuning of experimental variables would be possible to operate according to a designer’s policy.
Table 6.1 Scenarios for On-Water Experimentation interacting were common. voidance. Multi-encounter scenarios with 4-5 autonomous vehicles simultaneously
Table 6.1 Scenarios for On-Water Experimentation interacting were common. voidance. Multi-encounter scenarios with 4-5 autonomous vehicles simultaneously
characteristics in its own right: five simultaneous collision tracks of different headings
characteristics in its own right: five simultaneous collision tracks of different headings
rithm performance and not specific implementation of the rules of the road.
rithm performance and not specific implementation of the rules of the road.
Figure 6-5 This nominal on-water testing geometry allowed up to 5 autonomous vessels of varying speeds the ability to interact in successive, non-deterministic, non-canonical encounter geometries. The human-operated high-speed motorized kayak (shown in red) operated without track restrictions which allowed for fur- ther perturbation during the on-water Monte Carlo experimentation of Scenario D. The human also inserted non-compliant behavior to further stress the autonomous vessels.
Figure 6-5 This nominal on-water testing geometry allowed up to 5 autonomous vessels of varying speeds the ability to interact in successive, non-deterministic, non-canonical encounter geometries. The human-operated high-speed motorized kayak (shown in red) operated without track restrictions which allowed for fur- ther perturbation during the on-water Monte Carlo experimentation of Scenario D. The human also inserted non-compliant behavior to further stress the autonomous vessels.
ranges given the increased constraints on their operating environment. reduced by 37.5% of the open ocean scenario to allow the vehicles to accept close1
ranges given the increased constraints on their operating environment. reduced by 37.5% of the open ocean scenario to allow the vehicles to accept close1
Figure 6-7 Testing was performed using the autonomous vessels displayed here. High-speed and human interactions were achieved using the motorized human- operated kayak by Mokai.
Figure 6-7 Testing was performed using the autonomous vessels displayed here. High-speed and human interactions were achieved using the motorized human- operated kayak by Mokai.
Figure 6-8 The daytime testing range was marked with a buoy field and allowed for constrained vehicle testing. view of vehicle testing within the buoy field is shown in Figure 6-10.
Figure 6-8 The daytime testing range was marked with a buoy field and allowed for constrained vehicle testing. view of vehicle testing within the buoy field is shown in Figure 6-10.
Figure 6-9 Night testing was the primary setup and allowed for extensive use of large scale experiments. The three track pattern of Figure 6-2 is shown with the safety region surrounding the testing area.
Figure 6-9 Night testing was the primary setup and allowed for extensive use of large scale experiments. The three track pattern of Figure 6-2 is shown with the safety region surrounding the testing area.
Table 6.2. Number of On-Water Encounters Observed per Scenario of Section 3.3. gorithm. All other scenarios of Table 6.1 use the full CPA quantification algorithm
Table 6.2. Number of On-Water Encounters Observed per Scenario of Section 3.3. gorithm. All other scenarios of Table 6.1 use the full CPA quantification algorithm
Figure 6-11 This track data shows the perturbations of geometry possible with a long duration experiment using the starburst pattern of Figure 6-2. 6.3.1 Scenario A & B Results
Figure 6-11 This track data shows the perturbations of geometry possible with a long duration experiment using the starburst pattern of Figure 6-2. 6.3.1 Scenario A & B Results
the primary metric for comparison of algorithm performance under the five vehicle
the primary metric for comparison of algorithm performance under the five vehicle
igure 6-13 Example on-water objective functions with full CPA quantification. This snapshot of a point in time shows ownship’s objective functions corresponding to the beginning of the tracks displayed in Figure 6-12. The white dot at the center of each figure represents the center of the decision space. The pink dot represents the maximum value of each objective function. The top left figure shows the combined objective function for a desired transit to the east (mission objective function shown as Figure 6-3) with three simultaneous contacts of concern. Ownship’s optimal course-speed pair (a* of top left image) is depicted by the white arrow in each graphic. Ownship’s COLREGS collision avoidance objective functions (A - top right, B - bottom right, C - bottom left) at the same instant show compromised (non-red) values for A and B with an optimal value for C.
igure 6-13 Example on-water objective functions with full CPA quantification. This snapshot of a point in time shows ownship’s objective functions corresponding to the beginning of the tracks displayed in Figure 6-12. The white dot at the center of each figure represents the center of the decision space. The pink dot represents the maximum value of each objective function. The top left figure shows the combined objective function for a desired transit to the east (mission objective function shown as Figure 6-3) with three simultaneous contacts of concern. Ownship’s optimal course-speed pair (a* of top left image) is depicted by the white arrow in each graphic. Ownship’s COLREGS collision avoidance objective functions (A - top right, B - bottom right, C - bottom left) at the same instant show compromised (non-red) values for A and B with an optimal value for C.
Table 6.4 Mean CPA Range Normalized to Desired CPA{ for 5-Vehicle Scenarios scenarios as shown in Table 6.6. The full CPA quantification algorithm resulted in
Table 6.4 Mean CPA Range Normalized to Desired CPA{ for 5-Vehicle Scenarios scenarios as shown in Table 6.6. The full CPA quantification algorithm resulted in
Figure 6-14 The velocity obstacle and CPA methods were compared in Scenario A using (Ro, = R,,.¢)- The red circle indicates ranges that violated (R,,,i, = 0.5-Re%). col ~~ “p min col Experimentation using on-water encounters demonstrated that the full CPA method reduced the number of occurrences where a contact entered the red circle as shown in Table 6.6.
Figure 6-14 The velocity obstacle and CPA methods were compared in Scenario A using (Ro, = R,,.¢)- The red circle indicates ranges that violated (R,,,i, = 0.5-Re%). col ~~ “p min col Experimentation using on-water encounters demonstrated that the full CPA method reduced the number of occurrences where a contact entered the red circle as shown in Table 6.6.
‘able 6.5 Number of On-Water Experiments Performed for 5- Vehicle Scenarios Table 6.6 Percent of Pepa < Rmin Events for Scenario A & B 5-Vehicle On- Water Collision Avoidance Encounters
‘able 6.5 Number of On-Water Experiments Performed for 5- Vehicle Scenarios Table 6.6 Percent of Pepa < Rmin Events for Scenario A & B 5-Vehicle On- Water Collision Avoidance Encounters
Figure 6-15 Protocol scores were computed using the concepts of Sections 4.3. Scores ranged from 0-100.
Figure 6-15 Protocol scores were computed using the concepts of Sections 4.3. Scores ranged from 0-100.
Table 6.12 Standard Deviation of Protocol Score
Table 6.12 Standard Deviation of Protocol Score
Table 6.11 Mean Protocol Score
Table 6.11 Mean Protocol Score
On-Water Mean Safety Scores for Each Rule and Scenario
On-Water Mean Safety Scores for Each Rule and Scenario
Table 6.13 Mean Safety Score
Table 6.13 Mean Safety Score
Table 6.14 Standard Deviation of Safety Score
Table 6.14 Standard Deviation of Safety Score
ence with a system may be “prone to over trusting the automation” especially with
ence with a system may be “prone to over trusting the automation” especially with
Figure D-2 This still image represents the most meaningful part of Scenario 2 a: presented to the respondent. The actual survey included a moving image (Graphic: Interchange Format) from time of detection to just pass time of CPA.
Figure D-2 This still image represents the most meaningful part of Scenario 2 a: presented to the respondent. The actual survey included a moving image (Graphic: Interchange Format) from time of detection to just pass time of CPA.
Figure D-4 This still image represents the most meaningful part of Scenario 4 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.
Figure D-4 This still image represents the most meaningful part of Scenario 4 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.
Figure D-5 This still image represents the most meaningful part of Scenario 5 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.
Figure D-5 This still image represents the most meaningful part of Scenario 5 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.
Figure D-6 This still image represents the most meaningful part of Scenario 6 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.
Figure D-6 This still image represents the most meaningful part of Scenario 6 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.
Figure D-7 This still image represents the most meaningful part of Scenario 7 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.
Figure D-7 This still image represents the most meaningful part of Scenario 7 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.
‘igure D-8 This still image represents the most meaningful part of Scenario 8 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.
‘igure D-8 This still image represents the most meaningful part of Scenario 8 as presented to the respondent. The actual survey included a moving image (Graphics Interchange Format) from time of detection to just pass time of CPA.
Table D.1 Individual Respondent Data Before Controlling for Percent of Blame
Table D.1 Individual Respondent Data Before Controlling for Percent of Blame
Figure D-10 This table displays the percentage of correct answers to COLRI] rule numbers as first reported by Zekic in [99]. Image is Table 1 from [99]. iS) Table 1 The percentage of correct answers tailed assessment of the degree of contradiction is possible for real-world encounters
Figure D-10 This table displays the percentage of correct answers to COLRI] rule numbers as first reported by Zekic in [99]. Image is Table 1 from [99]. iS) Table 1 The percentage of correct answers tailed assessment of the degree of contradiction is possible for real-world encounters
Figure D-11 This chart shows the reasons for collisions with competency of the rules prevailing. Having a set of objective and standard evaluation metrics and algorithms may promote improved competency through more standardized at-sea or in-simulator testing. Image is originally from Maritime Accident Investigation Branch Report of 2004 and reproduced as Figure 5 in [102].
Figure D-11 This chart shows the reasons for collisions with competency of the rules prevailing. Having a set of objective and standard evaluation metrics and algorithms may promote improved competency through more standardized at-sea or in-simulator testing. Image is originally from Maritime Accident Investigation Branch Report of 2004 and reproduced as Figure 5 in [102].
‘igure D-12 This chart shows ignorance and disregard were the prevailing reasons that drivers failed to obey COLREGS. Image is from Syms 2002 and reproduced as Figure 6 in [102].
‘igure D-12 This chart shows ignorance and disregard were the prevailing reasons that drivers failed to obey COLREGS. Image is from Syms 2002 and reproduced as Figure 6 in [102].
Related topics:
EngineeringMechanical EngineeringComputer ScienceCollision AvoidanceObstacle Avoidance
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved