Commit Graph

4547 Commits

Author SHA1 Message Date
Alex Vandiver 7b95f38854 install-aws-server: Link to run-instances documentation. 2022-01-12 15:52:19 -08:00
Alex Vandiver 44b8321721 install-aws-server: Shell hygene. 2022-01-12 15:52:19 -08:00
Alex Vandiver 1522eeaebf install-aws-server: Configurable disk size. 2022-01-12 15:52:19 -08:00
Alex Vandiver 0288964884 install-aws-server: Default to encrypted gp3 disks. 2022-01-12 15:52:19 -08:00
Alex Vandiver f52ee561ae install-aws-server: Configurable availability zone. 2022-01-12 15:52:19 -08:00
Alex Vandiver 8e89097dc1 install-aws-server: Intentionally space-split security groups.
This allows for multiple security groups to be specified,
space-separated, in the config.
2022-01-12 15:52:19 -08:00
Alex Vandiver c66fd6a01a install-aws-server: Configurable IAM profile for hosts. 2022-01-12 15:52:19 -08:00
Alex Vandiver d9d3368553 install-aws-server: Remove ASG logic. 2022-01-12 15:52:19 -08:00
Anders Kaseorg 82a048004a test-locked-requirements: Rename may_be_setup_cache to maybe_set_up_cache.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-12 13:21:35 -08:00
Steve Howell dcffa17e29 test-js-with-node: Simplify failures for --coverage.
When you use nyc, its code instrumentation transforms
the code so that line numbers and columns no longer
make sense, and the long stack trace is likely to cause
more confusion than convenience.

We want to encourage a workflow where you debug your
node tests using the normal (and much quicker mode)
before running `--coverage`.
2022-01-08 10:10:57 -05:00
Mateusz Mandera 5939329485 do_delete_user: Add migration to fix bugged UserProfiles.
do_delete_users had two bugs:
1. Creating the replacement dummy users
with active=True
2. Creating the replacement dummy users with email domain set to
realm.uri, which may not be a valid email domain.
Prior commits fixed the bugs, and this migration fixes the pre-existing
objects.
2022-01-04 15:42:03 -08:00
Alex Vandiver 0628c5d817 bootstrap-awscli: Set -eu, for if this is run standalone.
The existing callsites of this are via `source` or being inline'd into
the startup of a new host; in both of these cases, the surrounding
script is already `set -eu`.  However, if run as a standalone tool, it
should also configure itself to catch checksum failures and other
problems.
2021-12-31 11:14:40 -08:00
Alex Vandiver 7fcd173ec3 aws: Upgrade awscli version. 2021-12-31 11:14:40 -08:00
Steve Howell cb21506ea7 node tests: Sort list of exempt files. 2021-12-30 11:36:52 -08:00
Steve Howell 0777cd73bb stream edit: Extract stream_subscribers_ui.js.
This is a fairly straightforward extraction.

It's good to test this with Iago, and then go into
Manage Streams and add/remove subscribers for a stream
like devel.

I copy/pasted two small functions that will soon
diverge from stream_edit. The get_stream_id function
will either use a module variable (since we're
generally only editing subscribers for one stream, and
we already have the singleton assumption with
`input_pill`) or a more strict CSS selector.  And then
get_sub_for_target depends on get_stream_id. We may not
always need full subs, anyway, and when we adapt some
of this code for creating streams, things are likely to
change.

I stopped exporting a couple functions that have no
callers outside of this module.

The main entry point for the module is
enable_subscriber_management.

We continue to export invite_user_to_stream and
remove_user_from_stream, which should possibly be just
pulled into their own module to lessen some
dependencies, but they don't have too much baggage,
since they just wrap channel calls.
2021-12-30 11:36:52 -08:00
Anders Kaseorg f9271098bf upload-release: Avoid appending to bytes in a loop.
Appending to bytes in a loop leads to a quadratic slowdown since
Python doesn’t optimize this for bytes like it does for str.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-29 16:50:08 -08:00
Alex Vandiver 74b804880e bootstrap-aws: Support aarch64. 2021-12-29 16:35:15 -08:00
Mateusz Mandera c5c3ab66d6 remote_server: Migrate RemoteZulipServer.uuid to be UUIDField.
Given that these values are uuids, it's better to use UUIDField which is
meant for exactly that, rather than an arbitrary CharField.

This requires modifying some tests to use valid uuids.
2021-12-28 10:11:34 -08:00
Anders Kaseorg 702ce071f4 python: Accept Optional[FrameType] in signal handlers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-28 09:31:55 -08:00
Anders Kaseorg 2ff3b51e03 test-documentation: Suppress Unicode Private Use Area validator warning.
sphinx_rtd_theme references the Font Awesome fa-link icon directly
with U+F0C1.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-22 16:55:31 -08:00
Alex Vandiver 71b56f7c1c puppet: process_fts_updates connects as nagios (or provided username).
It should not use the configured zulip username, but should instead
pull from the login user (likely `nagios`), or an explicit alternate
provided PostgreSQL username.  Failure to do so results in Nagios
failures because the `nagios` login does not have permissions to
authenticated the `zulip` PostgreSQL user.

This requires CI changes, as the install tests install as the `zulip`
login username, which allowed Nagios tests to pass previously; with
the custom database and username, however, they must be passed to
process_fts_updates explicitly when validating the install.
2021-12-14 14:48:53 -08:00
Mateusz Mandera 323f5dbba2 lint: Enforce consistent style of using transaction.atomic decorator.
When decorating a function, @transaction.atomic and
@transaction.atomic() are equivalent. We can add a linting rule to
enforce consistency.
2021-12-12 11:15:33 -08:00
Steve Howell 1db74fcf2f test-all: Call documentation tests again.
I accidentally checked in a change where I commented
out the test.
2021-12-11 12:46:44 -08:00
Steve Howell b2d83a8300 tests: Split out SingleUserExportTest.
This is mostly moving code, plus I now just
call shutil.rmtree directly.
2021-12-11 13:06:41 -05:00
Alya Abbott 0836b51738 policies: Update Kandra Labs policies pages. 2021-12-10 17:56:12 -08:00
Eeshan Garg 5aaeb1a432 use_cases: Rename /for/companies to /for/business. 2021-12-09 17:16:52 -08:00
Tim Abbott f8e35b5b36 tools: Document indentation policy. 2021-12-06 15:01:29 -08:00
Steve Howell 1509e7f355 templates: Always indent "each" blocks. 2021-12-06 14:58:29 -08:00
Steve Howell cbcfbf83ae check-tempates: Improve error message for missing end tag. 2021-12-04 15:35:24 -08:00
Steve Howell fb574431cb check-templates: Rewrite pretty_print (again).
It now does everything based on the tokens, rather
than walking the lines and trying to match up tokens
to lines.
2021-12-03 14:35:34 -08:00
Steve Howell 7e7b628054 check-templates: Parse code blocks properly. 2021-12-03 14:35:34 -08:00
Steve Howell ddfb4def4e check-templates: Improve error output.
I have looked at maybe ~100 errors in the last week as part
of fixing the tooling, and it's quite common to want to just
see what the improved file would look like.  Now I show the
desired output with line numbers.

I also try to encourage devs to scroll up, since newbies
often don't do that for some reason when confronted with
error output.

Finally, I add some color. I try to repeat myself without
color for certain things in case colors on certain
backgrounds are hard to read.

A fast way to test this is to just break up a long tag
into two lines.
2021-12-03 14:35:34 -08:00
Anders Kaseorg c5a8894dfb install-shfmt: Upgrade shfmt from 3.4.0 to 3.4.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-03 14:33:53 -08:00
Anders Kaseorg befb56f905 install-shellcheck: Upgrade ShellCheck from 0.7.2 to 0.8.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-03 14:33:53 -08:00
Aman Agrawal 25ee6a795e compose: Never hide send button.
`Press Enter to send` used to hide `Send` button, we remove that
behaviour.

We show the current state of `Enter` hotkey action via text below
`Send` button which can toggle behaviour on click.
2021-12-02 15:11:15 -08:00
Mateusz Mandera 72e5664031 semgrep: Enforce use of get_object_from_key for Confirmation fetching.
get_object_from_key should be used when trying to fetch a Confirmation
object. There are some places that need to make
Confirmation.objects.filter(...) queries, so we can't completely ban the
pattern, but we can ban .get(...) and
.filter(..., confirmation_key=..., ...).
2021-12-02 12:05:08 -08:00
Steve Howell 42061359a9 dead code: Kill off most of html_branches.
The code that was removed here was intended
for tooling that either never got built or
became obsolete.
2021-12-02 09:49:10 -08:00
Steve Howell 5a5dcd6962 minor: Streamline IGNORE_IDS in check-templates. 2021-12-02 09:49:10 -08:00
Steve Howell c0d72ba236 check-templates: Avoid duplicate tokenizing step.
Now we only tokenize the file once, and we pass
**validated** tokens to the pretty printer.

There are a few reasons for this:

    * It obviously saves a lot of extra computation
      just in terms of tokenization.

    * It allows our validator to add fields
      to the Token objects that help the pretty
      printer.

I also removed/tweaked a lot of legacy tests for
pretty_print.py that were exercising bizarrely
formatted HTML that we now simply ban during the
validation phase.
2021-12-02 09:49:10 -08:00
Steve Howell 0decfa8da0 check-templates: Remove prevent_dangling_tags.
This was a misfeature--it occasionally flagged
code that was ugly but necessarily ugly to
ensure proper rendering.
2021-12-02 09:49:10 -08:00
Steve Howell 6152bed378 check-templates: Prevent excessive newlines. 2021-12-02 09:49:10 -08:00
Steve Howell 6fc445ddd9 check-templates: Defer some indentation checks.
This accomplishes a few things:

    * lighten the load for the main validation loop
    * defer indentation checks until we are sure the author
      even knows how to match up tags
    * add some info to the Token objects that we may soon
      consume in our pretty-printer
2021-12-02 09:49:10 -08:00
Steve Howell f1fa9bff83 refactor: Extract tag_flavor(). 2021-12-02 09:49:10 -08:00
Steve Howell c6799c0903 refactor: Move logic for HTML_VOID_TAGS.
This sets up for the next commit.
2021-12-02 09:49:10 -08:00
Steve Howell c4b181a169 check-templates: Insist on 4-space indents.
We now complain about programmers who don't use
4-space indents in template files, rather than
letting the pretty printer fix them.

This is partly just to simplify the pretty printer
code (in future commits), but it also makes the
symptom more obvious to newbie developers. They
are probably just as able to react to the direct
error messages as they are able to figure out how
to read diffs from the pretty printer and grok
the --fix syntax. And once they learn the convention
and configure their editor, it should then be a
one time problem.
2021-12-02 09:49:10 -08:00
Steve Howell 2f0f27b841 templates: Fix minor whitespace errors.
(we also add validation)
2021-12-02 09:49:10 -08:00
Steve Howell a744e38e67 check-templates: Make parser more thorough (and faster).
We now create tokens for whitespace and text, such that you
could rebuild the template file with "".join(token.s for
token in tokens).

I also fixed a few bugs related to not parsing
whitespace-control tokens.

We no longer ignore template variables, although we could do
a lot better at validating them.

The most immediate use case for the more thorough parser is
to simplify the pretty printer, but it should also make it
less likely for us to skip over new template constructs
(i.e. the tool will fail hard rather than acting strange).

Note that this speeds up the tool by almost 3x, which may be
slightly surprising considering we are building more tokens.
The reason is that we are now munching efficiently through
big chunks of whitespace and text at a time, rather than
checking each individual character to see if it starts one
of the N other token types.

The changes to the pretty_print module here are a bit ugly,
but they should mostly be made irrelevant in subsequent
commits.
2021-12-02 09:49:10 -08:00
Steve Howell 2eac0560b2 check-templates: Explicitly skip comment tags. 2021-12-02 09:49:10 -08:00
Steve Howell 7f651b6aa2 refactor: Remove pseudo singleton end tags.
These aren't necessary any more.
2021-12-02 09:49:10 -08:00
Steve Howell 2f5ae1a54b check-templates: Be more flexible with inline tags.
We now only complain about inline tags having mismatched
indents if the author clearly tried to format them as
blocks.
2021-12-02 09:49:10 -08:00