This works around a bug in Django in handling the error case of a
client sending an inappropriate HTTP `Host:` header. Various
internal Django machinery expects to be able to casually call
`request.get_host()`, which will attempt to parse that header, so an
exception will be raised. The exception-handling machinery attempts
to catch that exception and just turn it into a 400 response... but
in a certain case, that machinery itself ends up trying to call
`request.get_host()`, and we end up with an uncaught exception that
causes a 500 response, a chain of tracebacks in the logs, and an email
to the server admins. See example below.
That `request.get_host` call comes in the midst of some CSRF-related
middleware, which doesn't even serve any function unless you have a
form in your 400 response page that you want CSRF protection for.
We use the default 400 response page, which is a 26-byte static
HTML error message. So, just send that with no further ado.
Example exception from server logs (lightly edited):
2017-10-08 09:51:50.835 ERR [django.security.DisallowedHost] Invalid HTTP_HOST header: 'example.com'. You may need to add 'example.com' to ALLOWED_HOSTS.
2017-10-08 09:51:50.835 ERR [django.request] Internal Server Error: /loginWithSetCookie
Traceback (most recent call last):
File ".../django/core/handlers/exception.py", line 41, in inner
response = get_response(request)
File ".../django/utils/deprecation.py", line 138, in __call__
response = self.process_request(request)
File ".../django/middleware/common.py", line 57, in process_request
host = request.get_host()
File ".../django/http/request.py", line 113, in get_host
raise DisallowedHost(msg)
django.core.exceptions.DisallowedHost: Invalid HTTP_HOST header: 'example.com'. You may need to add 'example.com' to ALLOWED_HOSTS.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File ".../django/core/handlers/exception.py", line 109, in get_exception_response
response = callback(request, **dict(param_dict, exception=exception))
File ".../django/utils/decorators.py", line 145, in _wrapped_view
result = middleware.process_view(request, view_func, args, kwargs)
File ".../django/middleware/csrf.py", line 276, in process_view
good_referer = request.get_host()
File ".../django/http/request.py", line 113, in get_host
raise DisallowedHost(msg)
django.core.exceptions.DisallowedHost: Invalid HTTP_HOST header: 'example.com'. You may need to add 'example.com' to ALLOWED_HOSTS.
Apparently, this sockjs.tornado logging code resulted in a lot of
buggy error emails whenever a Zulip browser tried to reconnect on a
new IP. I don't see an obvious way to suppress them from within
sockjs, but that might be a good follow-up issue.
Fixes#6959.
The comment is pretty self-explanatory. The fact that Google Compute
Engine has this problem does not impress confidence about their
product, but hopefully this is the only really dumb thing they do.
Fixes#4839.
The original "quality score" was invented purely for populating
our password-strength progress bar, and isn't expressed in terms
that are particularly meaningful. For configuration and the core
accept/reject logic, it's better to use units that are readily
understood. Switch to those.
I considered using "bits of entropy", defined loosely as the log
of this number, but both the zxcvbn paper and the linked CACM
article (which I recommend!) are written in terms of the number
of guesses. And reading (most of) those two papers made me
less happy about referring to "entropy" in our terminology.
I already knew that notion was a little fuzzy if looked at
too closely, and I gained a better appreciation of how it's
contributed to confusion in discussing password policies and
to adoption of perverse policies that favor "Password1!" over
"derived unusual ravioli raft". So, "guesses" it is.
And although the log is handy for some analysis purposes
(certainly for a graph like those in the zxcvbn paper), it adds
a layer of abstraction, and I think makes it harder to think
clearly about attacks, especially in the online setting. So
just use the actual number, and if someone wants to set a
gigantic value, they will have the pleasure of seeing just
how many digits are involved.
(Thanks to @YJDave for a prototype that the code changes in this
commit are based on.)
This endpoint is part of the old tutorial, which we've removed, and
has some security downsides as well.
This includes a minor refactoring of the tests.
Create a new custom email backend which would automatically
logs the emails that are send in the dev environment as
well as print a friendly message in console to visit /emails
for accessing all the emails that are sent in dev environment.
Since django.core.mail.backends.console.EmailBackend is no longer
userd emails would not be printed to the console anymore.
In 1.2.15 version of django-auth-ldap, the authenticate() function of
LDAPBackend takes username and password as keyword arguments. This
commit updates the code to match this change.
Fixes#6588
It turns out that very little code change is required to support
GitHub auth on mobile. Ideally, this would come with tests, though
the complicated part of the code path is covered by the Google auth
version. But writing a test for this would take a long time, and I
think it's worth having the feature now, so I'll be doing tests as a
follow-up project.
The main change here is moving SOCIAL_AUTH_FIELDS_STORED_IN_SESSION to
be with the other hardcoded settings, since it's not something that
makes sense for a sysadmin to change. But while we're at it, we also
group the overall social auth settings separately from the
GitHub-specific settings.
This isn't something that a user can ever modify, so it doesn't belong
in DEFAULT_SETTINGS. While we're at it, we align the appearance of
the email gateway in the docs with whether this setting in the docs
will be valid.
This will help identify the settings that need attention: either
to remove, or to document for server admins, or to just add a
comment to explain.
Identified with the following shell "one-liner" (one 313-char line
as I originally ran it; indentation added here for clarity):
perl -lne 'next unless (/^DEFAULT_SETTINGS/../\}\)?$/);
next unless (/'\''(.*?)'\''/);
print $1' \
zproject/settings.py \
| while read var; do \
echo -n "$var: "; \
(grep -lw "$var" zproject/{prod_settings_template,{dev,test}_settings}.py \
|| echo none) \
| sed s,zproject/,,g \
| fmt -w1000; \
done
This doesn't yet do much, but it gives us a suitable place to
add code to customize how log messages are displayed, beyond what
a format string passed to the default formatter can do.
This should make it a little easier to understand our logging config
and make changes to it with confidence.
Many of these items that are now redundant used to be required when we
were setting disable_existing_loggers to True (before 500d81bf2), in
order to exempt those loggers from being cleared out. Now they're not.
One bit of test code needed a tweak to how it got its hands on the
AdminZulipHandler instance; it can do it from the list on the root
logger just as well as on the `django` logger.
Most of the paths leading through this except clause were cut in
73e8bba37 "ldap auth: Reassure django_auth_ldap". The remaining one
had no test coverage -- the case that leads to it had a narrow unit
test, but no test had the exception actually propagate here. As a
result, the clause was mistakenly cut, in commit
8d7f961a6 "LDAP: Remove now-impossible except clause.", which could
lead to an uncaught exception in production.
Restore the except clause, and add a test for it.
Since we made ZulipLDAPException a subclass of
_LDAPUser.AuthenticationFailed, the django-auth-ldap library already
handles catching it and returning None.
This fixes missing test coverage in this function introduced by
73e8bba379.
This was giving a couple of lines of logs on every normal,
successful connection -- clearly a job for DEBUG, but emitted
on INFO. Quiet it down.
Fixes#6674.
The main `authenticate` method in the django_auth_ldap package logs a message
at `exception` level if it passes through an exception it wasn't expecting.
Sensible practice, but we'd been passing through just such an exception for
any kind of routine authentication failure. After we recently stopped suppressing
an arbitrary subset of loggers with `disable_existing_loggers`, these started
showing up noisily, including in tests.
So, make our exceptions expected. Just like our own code, the upstream code
raises exceptions of a particular type for routine auth failures, and catches
them and just returns None. We make our type derive from that one, so as to
just piggyback on that behavior.
Fixes an issue reported in a comment to #6674.
The `disable_existing_loggers` option to the `logging.config` module
turns on a rather complicated behavior of disabling some, but not all,
loggers that might have been already configured when the call to
`logging.config.dictConfig` or `logging.config.fileConfig` is made:
> This behaviour is to disable any existing loggers unless they or
> their ancestors are explicitly named in the logging configuration.
(https://docs.python.org/3/library/logging.config)
Turns out the only reason this is there is as a compatibility hack to
match the behavior of Python 2.4 and below. See the thread where the
new behavior was introduced: https://bugs.python.org/issue3136
Just as the author of the new behavior explains in that thread from
2008, the legacy behavior forces all logging configuration to be
awkwardly centralized in one place. That makes the code harder to
read, and it perennially causes confusion when a perfectly
normal-looking `logging.getLogger` call at the top level of one module
mysteriously has no effect, while that in another module works fine,
under the influence of the details of what gets imported when.
So, switch to the shiny new behavior of Python 2.5. Here LOGGING is a
Django setting which just becomes an argument to logging.config.dictConfig.
This may cause a few of the logfiles in ZULIP_PATHS to become active
that have been dormant for a long time.
This change means that almost every Zulip server out there will now be
using subdomains for every realm. There are a few complications noted
in the release notes.
This commit implements support for copying over static files
for all bots in the zulip_bots package to
static/generated/bots/ during provisioning. This directory
isn't tracked by Git. This allows us to have access to files
stored in an arbitrary zulip_bots package directory somewhere
on the system. For now, logo.* and doc.md files are copied over.
This commit should act as a starting point for extending our
macro-based Markdown framework to our bots/API packages'
documentation and eventually rendering these static files
alongside our webhooks' documentation.
The motivation for this is that we'll want to use the STATIC_ROOT
variable in this code in the upcoming commits.
While we're at it, we give it a proper section in the file.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.
Commit message and error text tweaked by tabbott.
Fixes#6543.
This commit enables user to authenticate with any attribute set in
AUTH_LDAP_USER_SEARCH given that LDAP_EMAIL_ATTR is set to an email
attributes in the ldap server. Thus email and username can be
completely unrelated.
With some tweaks by tabbott to squash in the documentation and make it
work on older servers.
This is necessary for tests the to pass with the new config_error code
path, which we don't want to trigger except on specific tests for
those features.
Since we're auto-detecting the value anyway, there's no reason it
can't be moved to DEFAULT_SETTINGS.
This lets us remove some clutter from the installation documentation.
This adds the authors to the Zulip repository on GitHub from
/authors/ along with re-styling the page to fit the same
aesthetic as /for/open-source/ and other product-pages.
The new endpoints are:
/json/mark_stream_as_read: takes stream name
/json/mark_topic_as_read: takes stream name, topic name
The /json/flags endpoint no longer allows streams or topics
to be passed in as parameters.
With this change, we get as far as printing the message
"APNS: Sending apple push notification to devices" to the
log when a recent TestFlight build of the app is due for
a notification, and then don't hit an exception. But
on the other hand I still don't get an actual notification
on my phone, so there's still some debugging to do.
Here are the functions in top_left_corner:
get_global_filter_li: pure code move
update_count_in_dom: simplifed copy of similar function in stream_list.js
update_dom_with_unread_counts: pure code move, split out from function
of same name in stream_list.js
delselect_top_left_corner_items: pure code move
handle_narrow_activated: pure code move + rename
handle_narrow_deactivated: pure code move, split out from from function
of smae name in stream_list.js
Greg Price
Tim Abbott
Brock Whittaker
Alena Volkova
Vishnu Ks
Umair Khan
rht
hollywoodno
Eeshan Garg
Juliana Bacelar
Vishnu Ks
julianasobreira
Supermanu
Robert Hönig
Steve Howell
Vaida Plankyte
Aditya Bansal
Greg Price
Jack Zhang