<@U016HSAA3RQ> <@U022FPLS0FP> Tapeout Jobs immedia...
# chipignite
u
@jeffdi @omla Tapeout Jobs immediately fail, and there is nothing in the logs (in fact, it won't even email me the job data when I request it). I tried three times, but all fail. Is there anything else I can do to debug / fix this? Is this a known issue?
Tapeout job id: 04dbeebe-452f-47b4-8fa7-33370c820657
o
@Uri Shaked we’ll look into it
u
thx!
o
@Uri Shaked can you retry and let me know if you are still facing issues ?
u
It still fails, but now I can at least see some logs
I can see a few errors in the logs, e.g.
Copy code
ERROR: unbound (ghost) cell=caravel_openframe
ERROR: unbound (ghost) cell=caravel_240476a0_fill_pattern
Found 2 total unbound (ghost) cells
caravel_openframe () leaf empty ghost
caravel_240476a0_fill_pattern () leaf empty ghost
or
Copy code
flattening chip boundary...
done.
step size = 70.0
llx=1.0 lly=1.0 urx=-1.0 ury=-1.0
ERROR: fom_density.drc: fatal error, chip_boundary bbox empty or malformed
x_cnt = -1
y_cnt = -1
ERROR: fom_density.drc: fatal error, x-count or y-count < 1: -1 -1
and in the magic logs:
Copy code
Copying output for cell caravel_240476a0_fill_pattern from /mnt/users_data/jobs/urish/tinytapeout-06/4176bc1a-4cb0-4888-aea0-d911d45ee865/outputs/caravel_240476a0_fill_pattern.gds
Calma output error:  Can't find GDS file "/mnt/users_data/jobs/urish/tinytapeout-06/4176bc1a-4cb0-4888-aea0-d911d45ee865/outputs/caravel_240476a0_fill_pattern.gds" for vendor cell "caravel_240476a0_fill_pattern".  It will not be output.
   Copying output for cell caravel_openframe from /mnt/users_data/jobs/urish/tinytapeout-06/4176bc1a-4cb0-4888-aea0-d911d45ee865/outputs/caravel_openframe.gds
Calma output error:  Can't find GDS file "/mnt/users_data/jobs/urish/tinytapeout-06/4176bc1a-4cb0-4888-aea0-d911d45ee865/outputs/caravel_openframe.gds" for vendor cell "caravel_openframe".  It will not be output.
   Generating output for cell caravel_240476a0
Ended: 03/14/2024 15:19:23
seems like some files are missing?
o
@Uri Shaked not sure, which magic log is this ?
u
The one I get when I click on "Request Job Data"
logs.zip
o
I am taking a look
u
thx
o
Hello @Uri Shaked, can you give your
tapeout
another shot ?
u
It works! thank you
Trying to push another revision, and getting an error - seems like the repo is out of disk space?
Copy code
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib64/python3.8/logging/__init__.py", line 1089, in emit
    self.flush()
  File "/usr/lib64/python3.8/logging/__init__.py", line 1069, in flush
    self.stream.flush()
OSError: [Errno 28] No space left on device
and now just getting:
Copy code
<mailto:git@repositories.efabless.com|git@repositories.efabless.com>: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
fatal: Could not read from remote repository.
(even though the same setup worked 3 days ago)
o
@Uri Shaked apologies for the inconvenience, I’ll look into it
1
@Uri Shaked please try again now
u
yup, looks like it's working
thanks!
@omla our tapeout job (7ee5ff78-4e80-4d81-8cae-1185efff9a4e / TT-20240421-0bc4d01) failed, but I couldn't find any error message in the logs.
can you please take a look? we can't figure out why it has failed
o
I believe putting 'caravel_core.v' under your RTL directory should work, that is a work around till this is investigated
u
Just an empty caravel_core.v ? where exactly?
Each iteration takes almost 6 hours (precheck + tapeout), so we'd rather get it right
o
'/verilog/rtl', yes the content doesn't matter
well I can't be 100% sure, either that or you can wait till tomorrow and we should have an update that should take care of this and you'd only need to rerun tapeout
1
u
Thanks, trying once with /verilog/rtl/caravel_core.v
Unfortunately, it didn't do the trick 😞
We'll have to wait for tomorrow then
o
Hello @Uri Shaked , please run another tapeout and report back the result at your earliest convenience
u
Trying now. Just tapeout - no need for precheck, right?
o
yes
1
u
Thanks! Now we just need to wait 2:30 hours to find out if it worked this time 🙂
Failed again, unfortunately
o
ok, thanks for letting me know, looking into it and hopefully will get back to you shortly
1
u
Hi! Any progress on this?
o
will inform you as soon as I've something
1
Hello @Uri Shaked, we’re able to identify why
tapeout(s)
have been failing and we’re hoping to land on a fix later today, sorry for the inconvenience.
1
u
Thanks for the update!
o
Hello Uri, an update was deployed to fix the issue above, please re-submit tapeout as soon as you're able to
u
Amazing - just submitted it
Happy Monday 🙂 Looks like "Request Job Data" is broken - when clicking on it, I get the green confirmation message saying "Successfully requested data for job", but the email never arrives (tried on Friday, and now again - no email arrived).
The job id was 6e24c098-5166-439d-937e-0bad79d4a253
Update: I tried again now, and got the email with the download links. Thanks for fixing!