Mike Gerwitz

Activist for User Freedom

path: root/bin
Commit message (Collapse)AuthorAgeFilesLines
* bin/tame: Fix runner output line clearingMike Gerwitz2022-01-281-1/+1
| | | | | The output was being omitted under certain conditions, meaning that users would have to look in the runlogs for errors.
* tame: tamed --help: Add missing closing quote to awk exampleMike Gerwitz2022-01-261-1/+1
* tamed: Fix --help and add another reporting exampleMike Gerwitz2022-01-201-1/+4
| | | | | $2 was not escaped and would fail expansion. I apparently did not run --help before committing. Shame on me.
* tamed: Add runtab and TAMED_RUNTAB_OUTMike Gerwitz2022-01-192-6/+142
| | | | | | | | | | | | | This provides logging that can be used to analyze jobs. See `tamed --help` for some examples. More to come. You'll notice that one of the examples reprents package build time in _minutes_. This is why TAMER is necessary; as of the time of writing, the longest-building package is nearly five and a half minutes, and there are a number of packages that take a minute or more. But, there are potentially other optimizations that can be done. And this is _after_ many rounds of optimizations over the years. (TAME was not originally built for what it is currently being used for.)
* tamed: Ignore SIGUSR2Mike Gerwitz2022-01-192-7/+0
| | | | | This was originally going to tell tamed to redraw the runner status line, but a different approach was taken.
* tamed: Add missing --report to helpMike Gerwitz2022-01-191-2/+4
| | | | Missing from previous commit.
* tamed --report and runner status line (TAMED_TUI)Mike Gerwitz2022-01-192-13/+295
| | | | | | | | | | | | | | | | | | | | | | This is something that I've wanted to do for quite some time, but for good reason, have been avoiding. `tamed --report` is fairly basic right now, but allows you to see what each of the runners are doing. This will be expanded further to gather data for further analysis. The thing that I was avoiding was a status line during the build to summarize what the runners are doing, since it's nearly impossible to do so from the build output with multiple runners. This will not only allow me to debug more easily, but will keep the output plainly visible to developers at all times in the hope that it can help them improve the build times themselves in certain cases. It is currently gated behind TAMED_TUI, since, while it works well overall, it is imperfect, and will cause artifacts from build output partly overwriting the status line, and may even occasionally clobber the PS1 by erasing the line. This will be improved upon in the future; something is better than nothing.
* Copyright year update 2021Mike Gerwitz2021-07-223-3/+3
* bin/tame (verify-runner): Add missing id paramMike Gerwitz2021-03-121-2/+3
| | | | | This was referencing a global $id, which is not the value we are interested in (and may not exist at all). Add the missing param.
* bin/tame{,d}: Fix assignments that lose exit codeMike Gerwitz2021-03-122-5/+8
| | | | Ensure that we fail if the command in the assignment fails.
* bin/tame: read {=>-r} where missingMike Gerwitz2021-03-121-2/+2
| | | | | While these should not be necessary in practice, there's no reason _not_ to do this.
* bin/tame: Remove unnecessary trailing backslashesMike Gerwitz2021-03-121-17/+17
| | | | | This was originally in a Makefile, long ago, where backslashes were actually needed.
* tamed: TAMED_JAVA_OPTS: New environment variableMike Gerwitz2020-08-193-1/+7
| | | | | This will be passed to dslc and then to the JVM. The intent is to permit fine-grained heap ratio tuning.
* Copyright year 2020 updateMike Gerwitz2020-03-063-3/+3
* bin/tamed: Fail without explicit DONEMike Gerwitz2020-01-211-0/+5
| | | | We want to fail e.g. on a JVM crash.
* tame: Create guard parent directoryMike Gerwitz2019-04-041-2/+5
| | | | It may not exist on certain systems (e.g. build containers).
* Parallel build supportMike Gerwitz2019-04-042-17/+220
| | | | | | | | | | | | | tamed was originally designed with support for parallel builds in mind, but I hadn't completed that work because we didn't have enough hardware that we'd benefit strongly from it. That has since changed. tamed will now spawn additional runners as needed to fulfill requests, which works around the issue of not knowing how many jobs GNU Make is going to try to do at once. There were a couple minor dependency fixes/workarounds for now in the Makefile, but otherwise everything appears to be working great.
* Copyright year simplification and update to Ryan Specialty GroupMike Gerwitz2019-02-073-3/+3
| | | | | | | | | This now uses year ranges, which I'll update annually. This also renames "R-T Specialty" to "Ryan Specialty Group". The latter is the parent company of the former. I was originally employed under the former when LoVullo Associates was purchased, by I now work for the parent company.
* bin/tame: Shift arguments after -vMike Gerwitz2018-12-041-1/+1
| | | | * bin/tame (main): Fix issue where -v did not shift arguments.
* tamed: Clarify usage output shell exampleMike Gerwitz2018-12-031-2/+2
| | | | * bin/tamed (usage): Clarify killing when run on a shell.
* tamed: Do not stall if TAMED_SPAWER_PID is runningMike Gerwitz2018-12-032-4/+35
| | | | | | | | | | | | | This will ensure that tamed does not stall while e.g. make is still running. This makes TAMED_STALL_SECONDS almost useless; maybe it'll be removed in future versions. * bin/tame (TAMED_SPAWNER_PID): Export variable. * bin/tamed (TAMED_SPAWNER_PID): New variable, default to PPID. (spawner-dead): New function. (stall-monitor): Use it. (usage): Update documentation. * build-aux/Makefile.am: Set TAMED_SPAWNER_PID to own id and export.
* dslc: Invoke with static rater pathMike Gerwitz2018-11-081-1/+15
| | | | | | | | | | This frees us from requiring a rater/ directory in the working directory. However, it is important that we continue using it if it exists, since there are additional things that haven't yet been moved into the tame repo. * bin/dslc.in: Provide path to rater/ directory. * src/current/src/com/lovullo/dslc/DslCompiler.java: Use provided rater/ path.
* bin/tame: Inherit TAME_CMD_WAITTIME from environmentMike Gerwitz2018-10-191-5/+16
| | | | | | | | | | * bin/tame (TAME_CMD_WAITTIME): Renamed from `RUNNER_CMD_WAITTIME'. Inherit from environment, default 3. (command-runner): Sleep for an additional TAME_CMD_WAITTIME seconds after requesting runner reload to give more time in case of high load. (verify-runner-ack): Rename variable. (usage): Document env var. * build-aux/Makefile.am: Export TAME_CMD_WAITTIME.
* bin/tame: Better runner re-tryMike Gerwitz2018-10-161-3/+7
| | | | | | | | | Try to re-post message, since the previous message will have already been read (otherwise the previous echo would have hung). * bin/tame (EX_STALLED): New exit code. (command-runner): Re-post message after stall. If unrecoverable, provide a more clear error and exit with EX_STALLED.
* Propagate TAMED_STALL_SECONDSv3.3.4Mike Gerwitz2018-10-161-0/+3
| | | | | bin/tame (TAMED_STALL_SECONDS): Export variable. build-aux/Makefile.am (TAMED_INSTALL_SECONDS): Export variable.
* tame{,d}: Reload runner when unresponsivev3.3.3Mike Gerwitz2018-10-162-6/+43
| | | | | | | | | | | | | This tries to be a bit more resilient in case a runner becomes unresponsive, rather than waiting for tamed to kill itself. * bin/tame (RUNNER_CMD_WAITTIME): New variable. (command-runner): Tell runner to reload if it does not respond in RUNNER_CMD_WAITTIME seconds. (verify-runner-ack): New function. * bin/tamed (mkfifos): Only keep stdin open. stdout isn't necessary, and may have actually been causing subtle issues. (spawn-runner): Support restarting dslc on SIGHUP.
* Integrate new compilation scripts, remove cqueue and Makefile.2Mike Gerwitz2018-10-112-25/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a major step toward normalcy---removing the kluge of a build process that was causing so many issues. Rather than echoing all operations to a queue file before passing it off to dslc, the new build scripts in `bin/' are used to invoke tame normally, as needed. This solves all of the current issues with things not rebuilding when they should. And, as a bonus, tab completion on targets works. Sorry this took so long. There wasn't much motivation until we hired so many people that are suffering from this. This does a few major things, along with some miscellaneous others: - Invoke bin/tame directly; - Merge Makefile.2.in into Makefile.am; and - Fix up some targets. * build-aux/Makefile.2.in: Delete file. Mostly merged with Makefile.am. * build-aux/Makefile.am: Add a bunch of new targets and definitions from Makefile.2.in. Modify all that previously used .cqueue to now invoke `$(TAME)' directly. Remove miscellaneous targets for trying to proxy targets to Makefile.2. (saneout, _go): Remove definitions. (.NOTPARALLEL): Add to prevent parallel builds. (ui/program.expanded.xml)[.version.xml]: Remove dependency for now. (clean): Also clean generated PHP files. Follow symlinks to clean core. This is still incomplete (does not clean all rate table stuff). (suppliers.mk)[xmlo_cmd]: Remove. See `gen-make' and `gen-c1make'. (lvroot)[summary-html]: New dependency. (kill-tamed, tamed-die): New targets (former alias of latter) to kill tamed. * build-aux/gen-c1make: Generate `$(TAME)' invocation. * build-aux/gen-make: Likewise. Remove `xmlo_cmd' output. Ignore recursive `tame' symlink (this can be removed once we clean `rater/' up. * build-aux/m4/calcdsl.m4 (TAME): Update description to reflect that it should now be the path to `bin/tame'. Adjust `AC_CHECK_FILE' lines accordingly. (tame_needed_ver): Remove. We have been in the same repo as TAME itself for quite some time. Remove associated code. (AC_CONFIG_FILES): Remove `Makefile.2'. * src/current/src/com/lovullo/dslc/DslCompiler.java (_DslCompiler)[compile]: Perform validation prefore `compile' command rather than a separate `validate' step. Remove `rm'. [compileSrc]: Stop echoing command. This was only necessary because of the previous Makefile klugery; now Make echoes on its own correctly.
* bin/: Server/client build scriptsMike Gerwitz2018-10-083-0/+465
These scripts allow the TAME compiler stack to be invoked naturally, rather than requiring the use of a Makefile today. This will not only allow users to more easily invoke the compiler, but will also allow us to invoke TAME naturally from Makefile and remove the klugery that has existed for so long. This users a server/client architecture in order to mitigate the startup cost of the JVM. More documentation will follow. Note that there are a bunch of symlinks in rater/---this is a transition step to allow the build to continue working as it did before, which relies on a directory structure that exists outside of this repository. This will be cleaned up in the future. * .gitignore (bin/dslc): Add ignore for generated file. * bin/dslc.in: New script to encapsulate Java invocation. * bin/tame: New script (client). * bin/tamed: New script (server). * configure.ac (JAVA_OPTS, DSLC_CLASSPATH, AUTOGENERATED): New variables for dslc.in. Output bin/dslc. * rater/README.md: Note that this symlink mess is temporary. * rater/c1map: New symlink for dslc assumptions. * rater/c1map.xsl: Likewise. * rater/calc.xsd: Likewise. * rater/compile.xsl: Likewise. * rater/compiler: Likewise. * rater/dot.xsl: Likewise. * rater/include: Likewise. * rater/link.xsl: Likewise. * rater/standalone.xsl: Likewise. * rater/summary.xsl: Likewise. * rater/tame: Likewise (warning: circular symlink). * src/current/src/com/lovullo/dslc/DslCompiler.java (_DslCompiler)[compile]: Output `DONE' lines.