Google Cloud new cluster generation failure. -

-

i have been trying create new cluster using both web ui , following command: gcloud dataproc clusters create cluster-2 --zone europe-west1-b --master-machine-type n1-standard-1 --master-boot-disk-size 50 --num-workers 2 --worker-machine-type n1-standard-1 --worker-boot-disk-size 50 --project <project-name>

the cluster consists of 1 master node , 2 worker nodes , pretty small cluster. virtual machines generated , running properly. however, cluster generation fails.

the error messages shown during cluster-generation point me file "dataproc-startup-script_output". error message have found error: "--max_wait_seconds" not port in file.

the number of vms have 5. single machines can created , run successfully. in recent past (few days ago), able create cluster no problems. cluster deleted however. there limit how many clusters 1 can create?

to summarize findings following separately email thread, in general if:

  1. it takes longer 10 minutes or fail, and
  2. you've changed project network settings @ all

then potential culprit network misconfiguration. in general, dataproc clusters require full internal ip networking access each other, , it's typical have firewall rule in google compute engine network opens of udp:1-65535,tcp:1-65535,icmp limited internal-ip "source ip range".

in case, project indeed missing necessary rule due minor typo/misconfiguration, default-allow-internal rule accidentally limited source ip range complete mask of /32.

if you're not doing advanced vpn configuration or cross-project networking, re-adding simple 10.0.0.0/8 udp:1-65535;tcp:1-65535;icmp firewall rule should work. if you're doing more advanced networking, you'll want restrict source range bit more specific, example if network's ipv4 range 10.128.0.0/16 should set "allow internal" firewall use 10.128.0.0/16 source range well.

adding firewall rules through cloud console provides convenience methods selecting applicable source ip ranges, convenient when have subnetworks messy enumerate manually.


Comments

Post a Comment

Popular posts from this blog

sql - invalid in the select list because it is not contained in either an aggregate function -

-
i can't figure out rewrite sql statement avoid error below msg 8120, level 16, state 1, line 26 column 'staffmember.staffmemberid' invalid in select list because not contained in either aggregate function or group clause. sql statment select (select [dbo].[udf_readstaffmemberbasic](sm.staffmemberid) xml path ('staffmembersummary'), root ('items'), type ), count(sm.staffmemberid) totalresults, ceiling(cast(count(sm.staffmemberid) decimal)/ @pagesize) totalpages staffmember sm inner join staffmembertoadditionalrole ar on sm.staffmemberid = ar.staffmemberid (sm.firstname '%' + @name + '%' or sm.surname '%' + @name + '%') , (@floorid null or sm.floorid = @floorid) , (@directorateid null or sm.directorateid = @directorateid) , (@additionalroleid null or ar.additionalroleid = @additionalroleid) order sm.staffmemberid offset @page * @pagesize rows fe...
Read more

Angularjs unit testing - ng-disabled not working when adding text to textarea -

-
i trying test angularjs component using karma, mocha, power-assert. have textarea , button in component, button disabled based on length of text in textarea. when run component in browser works perfectly. can't figure out how test functionality. here's bit of code. inquiryform.js function inquiryform($scope) { // plan add logic here } angular.module('myapp').component('inquiryform', { controller: inquiryform, controlleras: 'inquiryform', templateurl: 'inquiryform.html' }); inquiryform.html <div class="contact"> <div>thanks contacting us.</div> <form> <textarea ng-model="inquiryform.inquirytext" name=""></textarea> <input type="submit" value="send" ng-disabled="!inquiryform.inquirytext.length"> </form> </div> inquiryformtest.js describe('inquiry form ', () => { let $r...
Read more

How to start daemon on android by adb -

-
can me please, head swollen... i trying run daemon on android emu/device command: adb -s <device_name> shell su -c /dir/daemon <port_number> but nothink happens, , no errors! if do: adb -s <device> shell and form shell cmdline: su -c /dir/daemon <port_number> than work fine. try use shell-script , run: adb -s <device_name> shell sh su -c /dir/script.sh <port_number> and try generate script qt code port number , use: adb -s <device_name> shell sh su -c /dir/script.sh but not helped... problem if enter adb shell previous run daemon - work. problem on different device/emu/iso_image different command format not work. example: adb -s <device_name> shell su -c /dir/daemon <port_number> // work @ emu, not @ iso and vice: adb -s <device_name> shell su -c "/dir/daemon <port_number>" // work @ iso etc. everybody can answer what's matter? sorry english adb shell su -c ...
Read more